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FOREWORD 

This  report  was  prepared  by  the  Institute  for  Defense  Analyses  (IDA)  under 
Contract  MDA  903  84  C  0031,  Task  Order  T-L2-407.  The  cognizant  technical  officer  for 
this  task  is  Mr.  Gary  Boycan,  Assistant  Director,  Training  Systems  and  Technology, 
Office  of  the  Assistant  Secretary  of  Defense  (Force  Management  and  Personnel)/Training 
Policy  Directorate. 

This  paper  presents  a  conceptual  model  of  the  training  system  which  is  intended  to 
serve  as  a  systematic  aid  to  help  understand  the  managerial  relevance  of  some  of  the 
functional  elements  of  the  training  system  and  to  help  define  the  types  of  data  that  are  most 
useful  to  guide  the  operation  and  management  of  the  training  system.  The  training  system 
model,  in  conjunction  with  a  survey  of  the  available  training  performance  effectiveness 
data,  was  used  to  define  the  categories  or  types  of  data  that  are  most  needed  for  policy 
planning  by  senior  training  managers  at  the  DoD  and  Service  levels.  The  data  requirements 
were  integrated  into  a  proposed  master  plan  for  macro-level  training  data  base  development 
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EXECUTIVE  SUMMARY 

A  Macro-Level  Cost  Data  Base  has  been  established  at  the  Defense  Training  and 
Performance  Data  Center  in  Orlando,  Florida  to  assist  in  the  development  and  support  of 
^  training  policy.  The  Institute  for  Defense  Analyses  was  tasked  to  "provide  an  overview  of 

the  kinds  of  data  available  and  their  relevance  to  training  management  issues  and  to  the 
development  of  macro-level  training  data  bases."  This  paper  provides  a  rationale  for 
determining  the  types  of  data  most  relevant  to  establishing  training  policy.  It  also  provides 
a  series  of  recommendations  for  data  base  development  in  the  form  of  a  Macro-Level 
Training  Data  Base  Master  Plan  (Table  6,  p.  66). 

The  Training  Management  Model  (TMM)  is  central  to  the  development  of  a  rationale 
for  structuring  a  data  bank  needed  to  support  training  policy.  The  TMM  provides  a 

•  logically  consistent,  parsimonious  approach  to  the  development  of  a  responsive  macro-level 
training  data  base.  Based  on  a  small  set  of  assumptions  about  the  nature  of  both  the  macro- 
level  training  manager  and  the  training  system,  it  provides  systematic  guidance  in 
understanding  the  operation  of  the  training  system  from  the  perspective  of  the  macro-level 

d  training  manager.  The  TMM's  primary  purpose  is  to  serve  as  an  aid  in  the  design  and 

development  of  training  data  bases. 

The  TMM  consists  of  three  parts:  Cost-Production,  Performance  Effectiveness, 
and  Readiness  Evaluation.  The  Cost-Production  Function  is  primarily  quantitative  in 
®  nature.  It  is  concerned  with  the  cost  and  the  effectiveness  of  the  training  system  needed  to 

produce  the  numbers  of  trained  personnel  needed  to  meet  approved  organizational 
requirements.  This  is  the  standard  operating  condition  of  the  training  system.  Problems 
may  cause  a  departure  to  one  of  the  other  two  functions,  but  the  standard  baseline  is  the 

•  Cost-Production  Function. 

The  Performance  Effectiveness  Function  is  concerned  primarily  with  the  ability  of 
the  trainees  to  perform  well  on  the  job.  It  focuses  on  the  questions  of  how  effective  the 
training  is  in  terms  of  how  well  personnel  are  able  to  perform  on  the  job  after  they  complete 
training. 
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The  Readiness  Evaluation  Function  looks  at  the  quantitative  and  qualitative  effects 
of  training  on  combat  readiness.  Readiness  evaluations  can  take  either  of  two  forms: 
prescriptive  or  empirical.  The  prescriptive  form  evaluates  whether  all  the  individuals  and 
functional  units  have  completed  the  required  training  courses  and  events.  The  empirical 
form  provides  a  linkage  between  training  and  combat  or  combat-related  performance 
achievement  measures. 

The  TMM  and  a  survey  of  the  training  performance  evaluation  literature  provide  the 
basis  for  estimating  the  types  of  data  that  a  macro-level  training  manager  will  need  to 
formulate  and  support  a  wide  range  of  training  policies.  Data  base  options  and  strategies 
are  presented  in  association  with  each  of  the  three  operating  functions.  The  separate  sets  of 
data  base  options  and  strategies  are  summarized  and  integrated  into  a  Macro-Level  Training 
Data  Base  Master  Plan  (Table  6,  p.  66).  The  Master  Plan  presents  a  road  map  for  data  base 
development  It  also  provides  a  vehicle  forjudging  the  adequacy  of  current  data  bases. 
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I.  BACKGROUND 


Training  is  important  to  the  proper  functioning  of  the  military,  but  training 
managers  frequently  have  difficulty  in  competing  for  scarce  resources  in  a  highly 
competitive  financial  arena.  There  is  a  need  to  be  able  to  estimate  accurately  the  impact  of 
policy  decisions  on  baining  readiness  and  force  effectiveness.  If  the  relationship  between 
the  amount  and  type  of  training  and  effectiveness  in  combat  could  be  demonstrated  in 


quantitative  terms,  the  formulation  of  training  policy  would  be  relatively  simple  and  the 


competition  for  resources  would  be  minimal.  Training  programs  are  based  on  traditional 
practice  supported  by  the  trial-and-error  of  experience.  The  value  of  the  right  kinds  of 
training  was  presented  dramatically  by  Rowland  (1985): 

Many  accounts  of  operations  refer  to  the  problems  of  using  inexperienced 
troops;  there  is  a  clear  impression,  albeit  not  quantified,  that  inexperienced 
troops  do  not  perform  in  battle  as  well  as  those  who  have  prior  combat 
experience.  Apart  from  basic  training  they  have  been  expected  to  learn  'on- 
the-job'- -that  is  the  lucky  ones  leam-for  others  it  is  too  late,  so  it  is  hardly 
an  efficient  process. 


A  significant  portion  of  the  reason  that  training  has  a  problem  in  competing  for 
funds  stems  from  the  complexity  of  the  training  system  and  the  difficulty  in  relating 
changes  in  training  resources  to  specific  changes  in  training  outcomes.  Training  managers 
have,  in  essence,  been  required  to  argue  that  more  training  is  better  and  that  less  training  is 
worse  without  being  able  to  state  either  how  much  better,  better  is;  or  how  much  worse, 
worse  would  be.  Consequently,  although  training  receives  respect  for  being  an  essential 
component  of  military  operations,  it  often  loses  the  competition  for  funds  to  other  interests 
that  can  show  a  more  exact  relationship  between  budgeted  resources  and  results. 

!•  A.  MAGNITUDE  OF  TRAINING  MANAGEMENT  RESPONSIBILITIES 

I  Training  is  an  essential  element  in  the  operation  and  performance  of  the  Services.  It 

1  provides  the  knowledge  and  skills  necessary  to  convert  naive,  inexperienced  entry-level 

1  personnel  into  the  qualified  operators  and  technicians  needed  to  support  a  large,  technically 

I®  sophisticated  military  force.  The  training  process  requires  billions  of  dollars  for  individual 


I 
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training  and  additional  billions  for  collective  training.  The  size  and  complexity  of  the 
military  training  system  creates  a  major  management  challenge  which  creates  a  demand  for 
improved  management  tools  for  developing,  evaluating,  and  justifying  training  policy 
decisions. 

The  Services  conduct  both  individual  training  and  collective  training.  Individual 
training  prepares  individuals  to  perform  specific  duties  and  tasks,  while  collective  training 
prepares  groups  (teams,  crews,  etc.)  to  accomplish  tasks  required  of  the  group  as  an  entity. 
Most  individual  training  is  conducted  at  formal  schools,  while  most  collective  training  is 
accomplished  in  operational  field  units,  i.e.,  at  military  bases  and  in  field  exercises.  Both 
types  of  training  affect  training  readiness  (DoD,  1983) 

1.  Individual  Training 

The  magnitude  of  training  and  the  effects  of  personnel  turbulence  on  individual 
training  can  be  summarized  as  follows:  According  to  the  Military  Manpower  Training 
Report  FY  1988  (DoD,  1987),  individual  training  for  recruits,  officers,  specialized 
technicians  and  pilots  is  expected  to  cost  18.5  billion  dollars  in  Fiscal  Year  1988. 
Specialized  skill  training  accounts  for  the  largest  number  of  students  and  the  largest  total 
cost;  the  highest  cost  per  student  is  for  flight  training.  About  75  percent  of  this  student  load 
was  due  to  the  entry  of  new  personnel  into  the  military  service;  this  is  a  consequence  of  a 
significant  .lumber  of  relatively  short  military  careers  where  as  much  as  one-quarter  of  the 
entire  military  force  may  leave  each  year.  When  we  combine  the  number  of  man-years  of 
students  and  instructors,  we  see  that  12  percent  of  all  military  man-years  are  always 
assigned  to  schools. 

2.  Collective  Training 

Both  the  collective  training  process  and  its  effects  are  difficult  to  analyze  because 
they  encompass  the  full  spectrum  of  tasks  performed  by  teams  and  larger  functional  units. 
However,  like  individual  training,  collective  training  activities  require  a  significant 
investment  of  resources.  Pellicci  (1985)  indicates  that  collective  training  is  the  most 
expensive  type  of  training  in  terms  of  total  recurring  operating  and  support  costs. 
However,  the  costs  of  collective  training  are  not  known  precisely  at  present 

The  Services  do  not  budget  for  unit  training  as  a  separate  activity.  The 
Services'  operations  and  maintenance  budgets  include  a  line  item  that 
provides  funds  for  unit  training;  this  line  item  a'.so  includes  funds  for 
operations  and  some  support  (GAO,  1986). 
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3.  Readiness  Training 

The  relationship  between  readiness  and  resources  is  an  important  question,  but  one 
without  readily  available  answers.  This  problem  is  one  which  applies  to  defining  the 
relationship  between  resources  and  combat  readiness,  in  general,  and  training  readiness,  in 
particular.  The  DoD  Force  Readiness  Report  for  FY  1986  (DoD,  1985)  states: 

It  is  reasonable  to  inquire  about  the  relationships  of  resource  expenditures  to 
the  subordinate  readiness  indicators  and  to  wonder  about  the  cost  of 
increasing  readiness.  Efforts  to  respond  to  this  inquiry  are  grouped  under 
the  rubric-relating  resources  to  readiness.  As  with  many  reasonable 
questions  the  answer  is  quite  difficult  (p.  4). 

Later  the  same  report  concludes: 

Thus,  while  we  know  many  resources  that  go  into  readiness,  it  is  not 
feasible  at  this  point  to  accurately  estimate  just  how  much  the  readiness 
indicators  will  change  as  the  result  of  specific  resource  expenditures. 

(p.  5). 

B .  CONSTRAINTS  ON  TRAINING  MANAGEMENT 

The  macro-level  training  manager  is  positioned  at  the  highest  level  of  the  DoD  or 
Service  training  management  organization.  From  this  position,  the  training  system  is 
shaped  through  the  use  of  budgetary  powers  either  to  approve,  support,  and  defend  or 
disapprove,  resist,  and  oppose  the  proposed  budgets  for  some  part  of  the  training  system. 
The  macro-level  training  manager's  powers  to  influence  the  budget  are  constrained  by  the 
need  to  respond  to  the  training  requirements  imposed  by  external  forces  such  as  weapon 
system  development  and  personnel  policies. 

Personnel  policies  and  weapon  programs  are  major  forces  in  shaping  the  training 
establishment  An  example  of  the  interrelationship  between  personnel  and  training  policies 
was  presented  in  the  Department  of  Defense  Readiness  Report  for  FY  1984  (DoD,  1983). 
The  Marine  Corps  structure,  which  calls  for  a  high  proportion  of  junior  personnel,  requires 
the  training  of  numerous  replacements  each  year  in  order  to  maintain  the  force  at  trained 
strength.  The  continuing  influx  of  new  personnel  and  the  continuing  loss  of  trained 
personnel  serve  to  define  the  quantity  and  types  of  training  needed.  The  training  managers 
must  decide  how  to  meet  the  required  training  goals. 

Other  personnel  policies  and  the  effects  of  those  policies  contribute  to  a  continuing 
state  of  instability  as  personnel  are  promoted,  rotated,  released,  and  retired.  Unit  stability 
or  the  lack  of  it  places  a  significant  burden  on  collective  training.  (The  effects  of  these 
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personnel  policies  are  probably  intensified  by  the  fact  that  personnel  tend  to  forget 
previously  learned  procedures.)  An  army  battalion  that  completes  a  field  exercise  this  year 
will  in  many  respects  be  a  different  organization  when  it  is  scheduled  for  the  same  exercise 
next  year. 

C.  NEED  FOR  MACRO-LEVEL  TRAINING  DATA  BASES 

In  recognition  of  the  need  to  improve  the  supportability  of  training  resource  needs, 
a 


1982  Defense  Science  Board  study  group  recommended  that  DoD  set  up  a 
centralized  data  and  analysis  center  fix'  the  defense  training  community.  The 
DSB  concluded  that,  while  good  training  data  appeared  to  exist,  the 
information  was  not  readily  available  and  was  not  always  in  a  form  best 
suited  to  the  needs  of  training  managers.  (Sicilia,  1985) 

D.  ANALYTICAL  OBJECTIVES 

The  tasking  for  this  project  included  the  following  goals: 

1 .  Provide  an  overview  of  the  kinds  of  data  available  and  their  relevance  to 
training  management  issues  and  to  the  development  of  macro-level  training  data 
bases. 

2.  Identify  the  data  required  to  evaluate  several  illustrative  and  representative 
policymaking  situations. 

3.  Collect  and  evaluate  the  data  required  for  at  least  three  prototype  policy 
decisionmaking  situations  to  demonstrate  the  range  of  policy  decisions  that  can 
be  supported  by  the  Macro-Level  Cost  Data  Bases. 

1 .  Objective  of  This  Paper 

This  paper  is  the  first  phase  of  an  analytical  effort  to  determine  the  types  of  data 
required  to  support  the  needs  of  OSD  and  the  Services  for  formulating  and  supporting 
training  policy  decisions  and  to  assess  the  adequacy  of  current  training  data  bases.  It 
focuses  primarily  on  the  first  of  the  three  goals  noted  above.  The  first  goal  can  be  stated  in 
the  form  of  two  questions: 

1 .  What  types  of  data  should  the  Macro-Level  Cost  Data  Bases  contain? 

2.  Do  the  current  data  bases  contain  the  data  needed  to  support  training  policy 
decisions? 
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In  order  to  respond  to  these  questions  it  will  be  necessary  first  to  specify  the 
management  needs  to  be  satisfied  by  the  macro-level  training  data  bases.  After  determining 
the  kinds  of  information  most  likely  to  be  needed  by  training  managers,  it  will  be  necessary 
to  assess  whether  the  needed  information  is  available.  These  prerequisites  lead  to  the 
following  formulation  of  the  current  work:  (1)  develop  a  conceptual  model  of  the  training 
system  to  serve  as  a  guide  for  identifying  the  types  of  data  most  needed  by  macro-level 
training  managers;  and  (2)  determine  the  availability  of  the  required  data  and  test  the  validity 
of  the  conceptual  model  of  the  training  system  through  a  series  of  case  studies  of 
representative  training  policy  questions.  This  paper  will  present  a  conceptual  model  of  the 
training  system  from  the  training  manager's  perspective.  Based  upon  the  model  and  a 
summary  of  some  of  the  relevant  data,  some  strategies  and  options  for  developing  the 
required  data  bases  will  be  presented.  The  results  of  the  case  studies  will  be  presented  in 
subsequent  papers. 

2.  Objectives  of  Further  Work 

Further  work  will  address  the  second  and  third  objectives  of  this  project.  The 
utility  of  current  data  bases  for  the  macro-level  training  manager  will  be  evaluated  through 
the  use  of  a  set  of  representative  policy  case  studies  that  demonstrate  and  assess  the  ability 
of  the  data  bases  to  support  cost-effectiveness  analyses  of  training. 
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n.  scope 


The  basic  thesis  guiding  the  preparation  of  this  paper  was  that  a  description  and 
understanding  of  our  military  training  system,  the  types  of  training  effectiveness  data,  and 
the  utility  of  these  data  for  the  macro-level  training  manager  is  a  necessary  first  step  in  the 
process  of  identifying  the  specific  cost,  effectiveness,  and  cost-effectiveness  data  and  data 
base  architectures  needed  to  support  training  policy  decisions. 

The  conceptual  model  of  the  training  system  presented  here  is  intended  to  provide  a 
systematic  description  of  some  of  the  functional  elements  of  the  training  system  and  to  help 
define  the  types  of  data  that  are  most  useful  for  the  operation  and  management  of  the 
training  system.  The  Training  Management  Model  (TMM)  describes  the  operation  of  the 
military  training  system  and  specifies  the  major  training  effectiveness  data  that  are  needed 
from  a  macro-level  management  perspective.  The  TMM  and  its  three  functioning  modules 
were  used  to  define  the  categories  or  types  of  data  that  are  required  by  macro-level  training 
managers  at  the  DoD  and  Service  levels.  The  data  requirements  were  integrated  into  a 
proposed  set  of  strategies  and  options  for  the  development  of  the  required  macro-level 
training  data  bases. 
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III.  TRAINING  MANAGEMENT  MODEL 


There  are  a  number  of  possible  approaches  to  building  macro-level  training  data 
bases.  One  approach  would  be  to  try  to  gather  all  of  the  available  training  data  and  then 
figure  out  how  to  apply  them  to  solving  a  variety  of  policy  level  training  problems. 
However,  given  the  massive  amount  of  data  that  could  be  collected,  this  approach  would  be 
very  expensive  and  cumbersome  without  being  able  to  guarantee  responsiveness  to  training 
manager  needs.  A  second,  somewhat  better  approach  would  be  simply  to  ask  macro-level 
training  managers  to  indicate  what  kinds  of  information  are  needed  and  to  build  the  data 
base  on  the  basis  of  specific  requests.  This  approach  is  more  highly  focused,  more 
economical,  and  is  clearly  responsive  to  the  managers'  needs.  For  some  highly  repetitive 
problems,  this  approach  is  highly  satisfactory.  However,  in  many  instances,  this  approach 
will  produce  data  bases  which  can  respond  to  last  year’s  problems  this  year  and  this  year's 
problems  next  year  or  at  some  undetermined  future  time.  A  third  approach  would  develop 
the  training  data  bases  on  the  basis  of  a  conceptual  model  of  the  functioning  of  the  training 
system  from  the  macro-level  training  manager's  perspective.  Like  the  second  approach  the 
manager's  needs  are  the  primary  driving  force  of  the  data  base  development.  However, 
since  the  model  provides  a  more  global  perspective,  the  data  bases  will  be  responsive  to  a 
wider  ranges  of  problems  and  a  much  greater  ability  to  respond  to  the  training  manager's 
future  needs  can  be  built  into  the  architecture  and  content  of  the  data  bases. 

Thus,  the  framework  for  a  Training  Management  Model  has  evolved  with  the 
benefits  of  insights  provided  by  reviewing  the  significant  data  base  development  efforts  at 
the  Defense  Training  and  Performance  Data  Center,  discussions  with  a  macro-level  training 
manager,  and  a  survey  of  the  training  performance  evaluation  literature.  The  TMM 
represents  an  initial  effort  to  represent  the  training  system  from  the  macro-level  training 
manager's  perspective.  As  such,  the  model  is  intended  to  serve  as  a  systematic  aid  to  help 
understand  the  managerial  relevance  of  some  of  the  functional  elements  of  the  training 
system  and  to  help  define  the  types  of  data  that  are  most  useful  for  its  operation  and 
management.  Hopefully,  this  model  will  serve  its  primary  purpose  of  aiding  in  the  design 
and  development  of  training  data  bases  and  will  also  serve  to  stimulate  the  development  of 


more  sophisticated  models  of  the  training  system  that  could  be  used  to  further  advance 
training  management  capabilities. 


A.  ASSUMPTIONS,  CONCEPTS,  AND  DEFINITIONS 

1.  Macro-Level  Training  Manager 

The  model  building  effort  begins  with  the  basic  assumption  that  the  macro-level 
training  manager  seeks  to  respond  to  approved  national  defense  needs  by  providing  ade¬ 
quately  trained  personnel,  on  time,  and  for  an  acceptable  cost.  It  is  further  assumed  that  the 
size  of  the  training  budget  and  its  allocation  directly  determine  training  capabilities  and 
capacities  which  in  turn  directly  impact  the  unit  costs  and  quality  of  the  training  output. 
The  macro-level  training  manager  uses  budgetary  powers  to  manage  training  capabilities 
and  capacities  as  means  of  achieving  the  goal  of  providing  adequately  trained  personnel,  on 
time,  and  for  an  acceptable  cost. 

2.  Conversion  of  Resources  to  Training  Products 

A  clearly  visible  linkage  between  the  requirements  generated  by  technical  and 
organizational  systems  and  training  products  should  make  it  possible  to  use  the  training 
data  bases  for  formulating  and  supporting  policy  decisions  in  a  precise  and  quantitative 
manner.  From  the  macro-level  training  management  point  of  view,  money  is  the  common 
denominator  of  the  various  components  of  a  training  system.  It  can  be  converted  into 
various  training  resources  in  the  form  of  student  and  instructor  time,  training  facilities,  and 
equipment.  The  process  of  converting  resources  into  training  activities  and  products  is 
schematically  represented  in  Figure  1.  Resources  used  for  training  are  not  available  for 
other  Service  needs,  such  as  aircraft,  ships,  tanks,  or  weapons. 

The  training  process  converts  an  objective,  tangible  object  (money)  into  a 
subjective,  intangible  quality  (combat  readiness).  However,  as  part  of  the  process,  the 
training  system  produces  many  types  of  numbers  related  to  the  consumption  of  resources 
and  the  production  of  trained  personnel. 
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Figure  1.  The  Conversion  of  Resources  into  Training  Activities  and  Products 
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3 .  Definitions 


a.  Cost-Effectiveness  of  Training 

Before  the  cost-effectiveness  of  training  programs,  methods,  or  devices  can  be 
determined,  training  products  need  to  be  defined  and  some  measure(s)  of  training 
effectiveness  must  be  identified  and  assessed.  The  evaluation  of  training  effectiveness, 
costs,  and  cost-effectiveness  represents  a  significant  problem.  A  recent  review  of  the 
economic  issues  in  cost-effectiveness  analyses  of  military  skill  training  by  Henry  Solomon 
(1986)  made  the  following  observation: 

A  survey  of  the  literature  revealed  several  major  difficulties  in  evaluating  the 
cost-effectiveness  of  alternative  training  procedures  and  technologies.  One 
major  difficulty  is  the  assessment  and  measurement  of  training 
effectiveness.  A  second  is  the  lack  of  information  on  job  performance 
resulting  from  training. 

b.  Training 

Training  means  many  things  to  many  people.  In  order  to  avoid  misunderstanding, 
it  seems  appropriate  to  specify  the  conceptual  approach  to  training  used  in  this  paper. 
Training  will  be  considered  as  something  done  to  personnel  (i.e.,  a  "treatment")  to  improve 
their  ability  to  perform  some  set  of  required  military  tasks  (i.e.,  the  "requirement"). 
Training  is  primarily  reactive.  The  characteristics  of  personnel  and  systems  design 
combine  to  determine  the  quantity  and  quality  of  training  needed  to  assure  the  minimum 
acceptable  performance  levels  of  specified  weapon  and  support  systems.  This  paper  seeks 
to  define  training  data  characteristics  and  to  provide  a  rationale  that  will  assist  macro-level 
managers  in  the  process  of  organizing  and  using  training  data  bases  to  support  training 
policy  decisions. 

c.  Performance  Measurement 

Performance  measures  to  assess  the  effectiveness  of  training  will  be  considered 
within  several  broad  categories.  First,  is  the  effectiveness  of  the  training  system  to  produce 
adequately  trained  individuals  and  units  in  a  reasonable  time  for  an  acceptable  cost. 
Second,  is  the  assessment  of  individual  performance  within  the  training  and  as  it  influences 
the  operational  units.  Finally,  is  the  assessment  of  the  effects  of  training  on  collective 
performance  measures  assessing  either  achievement  in  combat  or  probable  contributions  to 
combat  readiness. 
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B .  OPERATION  OF  THE  TRAINING  MANAGEMENT  MODEL 


1 .  Basic  Dynamics 

The  Training  Management  Model  (TMM),  shown  in  Figure  2,  shows  the  relations 
between  the  following  components  that  affect  the  training  manager 

1 .  requirements 

2.  resources 

3 .  type  of  training 

a.  individual 

b.  collective 

4.  performance  evaluation/quality  control 

5 .  readiness  status. 

The  training  system  is  driven  by  military  requirements.  The  requirements  are  based  on 
national  strategy,  perceived  threat,  the  character  and  needs  of  present  and  projected 
platforms  and  weapons  systems,  and  personnel  quantity,  quality,  and  turbulence.  The 
results  of  the  training  provide  feedback  which  also  serves  to  modify  the  requirements.  This 
complex  of  external  and  internal  factors  serves  to  establish  the  magnitude  of  required 
training  and  the  composition  of  the  training  system. 

Training  resources  are  allocated  and  the  needed  individual  and  collective  training 
programs  are  conducted  to  meet  the  goals  established  by  the  requirements.  The 
requirements  specify  the  number  of  personnel  to  be  trained,  the  types  of  technical 
specialties  needed,  the  individual  and  collective  training  to  be  accomplished.  "Training 
requirements  are  determined  by  estimating  the  probable  future  attrition  of  required  levels 
and  skill  mix  of  personnel  due  to  completion  of  enlistments,  retirements,  and  the  need  for 
increases  in  personnel  levels  due  to  increases  and/or  changes  in  weapon  systems  and  force 
structures."  (Solomon,  1986) 

2 .  Structural  Components  of  the  Model 

The  macro-level  training  manager  must  be  concerned  with  three  major  functions 
addressed  by  the  training  system: 


I 
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Figure  2.  Training  Management  Model 
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1 .  the  cost  and  production  aspects  of  training, 

2 .  the  effectiveness  of  training, 

3 .  level  of  readiness  resulting  from  training. 

The  following  discussion  shows  how  these  functions  are  handled  by  the  TMM.  In  the 
following  chapters,  each  of  the  major  functions  of  the  TMM  will  be  discussed  more  fully  in 
conjunction  with  examples  of  the  available  training  effectiveness  data  and  the  presentation 
of  a  set  of  proposals  for  the  development  of  macro-level  training  data  bases.  The  three 
major  training  management  functions  represented  in  the  Training  Management  Model  are 
shown  in  Figure  3.  These  functions  represent  basic  ways  that  the  macro-level  training 
manager  interacts  with  the  training  system.  The  basic  functioning  of  the  overall  training 
system  does  not  change,  as  the  manager's  orientation  and  data  needs  change  to  respond  to 
various  specific  problems. 

a.  Cost-Production  Function 

The  Cost-Production  function  of  the  TMM  describes  the  normal  operation  of  the 
training  system.  This  is  represented  by  a  continuous  feedback  loop  consisting  of  the 
following  four  elements:  Requirements,  Resources,  Individual  Training,  and  Collective 
Training.  Based  upon  approved  sets  of  requirements,  resources  in  the  form  of  budgets, 
facilities,  and  personnel  are  channeled  into  individual  and  group  training  programs  which 
ultimately  serve  to  fulfill  or  modify  the  existing  requirements.  The  Cost-Production 
function  depends  upon  two  assumptions:  (1)  the  training  courses  and  events  are  effective 
in  conveying  the  necessary  information  and  skills  to  the  personnel  involved,  and 
(2)  trained  personnel,  teams,  and  units  advance  the  state  of  readiness  and  potential  combat 
effectiveness  of  the  armed  forces.  When  these  assumptions  have  questionable  validity,  the 
training  manager  needs  to  obtain  appropriate  performance  or  readiness  data. 

The  Cost-Production  function  is  primarily  quantitative  in  nature.  It  is  concerned 
with  costs  and  the  ability  of  the  training  system  to  produce  the  numbers  of  trained 
personnel  needed  to  meet  approved  organizational  requirements.  The  manager's  concern  is 
to  provide  the  required  number  of  trained  personnel  on  schedule  and  within  budget.  Except 
when  the  training  system  fails  to  provide  an  acceptable  product  in  te.  ms  of  the  quantity  and 
quality  of  trained  personnel,  this  is  the  standard  operating  condition  of  the  training  system. 
The  bulk  of  the  macro-level  training  managers  information  requests  will  be  generated  from 
within  this  functional  module  of  training  system  operation. 
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b.  Performance  Effectiveness  Function 


The  Performance  Effectiveness  function  of  the  TMM  supplements  the  Cost- 
Production  function  in  several  ways:  (1)  within  training  courses,  individual  performance 
data  (e.g.,  test  scores)  support  the  training  quality  control  process  and  provide  the  basis  for 
deciding  whether  an  individual  should  continue  the  next  phase  of  training,  graduate, 
recycle,  or  attrite,  and  (2)  data  on  group  performance  in  weapon  systems  (training  or 
operational)  provide  a  basis  for  evaluating  the  effectiveness  of  training  courses  or 
techniques.  A  positive  relationship  between  training  and  readiness  and  potential  combat 
effectiveness  is  implicitly  assumed.  The  Performance  Effectiveness  function  in  Figure  3 
consists  of  five  elements:  Requirements,  Resources,  Individual  Training,  Collective 
Training,  and  Performance  Evaluation/Quality  Control.  There  is  a  continuous  feedback 
function  between  the  Individual  Training  element  and  the  Performance  Evaluation/Quality 
Control  element.  However,  only  a  possible  feedback  function  is  represented  between 
Collective  Training  and  the  Performance  Evaluation/Quality  Control  element 

This  function  is  concerned  primarily  with  the  ability  of  the  trainees  to  perform  well 
on  the  job.  It  is  concerned  with  the  quality  of  training  and  more  importantly  with  the 
quality  of  the  performance  skills  of  trained  personnel.  It  focuses  on  the  questions  of  how 
good  the  training  is  or  how  well  personnel  are  able  to  perform  on  the  job  after  the 
completion  of  training.  This  is  one  of  the  most  commonly  conceived  notions  of  training 
effectiveness.  Although  data  on  effectiveness  (test  scores  or  job  performance  evaluations) 
can  have  a  dramatic  impact  upon  training  policy,  most  of  these  data  needs  are  formulated, 
collected,  and  used  at  the  individual  training  (micro)  level  and  have  only  negligible  impact 
on  policy-level  decisionmaking. 

It  is  interesting  to  note  that  while  the  large  amounts  of  performance  data  collected 
within  training  courses  are  essential  for  the  operation  of  the  training  system,  they  may  be  of 
little  or  no  value  to  the  macro-level  training  manager.  However,  group  data  of  performance 
within  systems  can  be  very  useful  in  assessing  the  impact  of  training  courses  or  of 
alternative  training  techniques  upon  performance.  This  type  of  data  may  be  used  to  either 
support  or  resist  changes  in  training  programs  or  techniques.  Training  changes  based 
either  on  performance  quality  or  cost-effectiveness  impact  budgets  and  are  of  value  to  the 
macro-level  training  manager. 
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c.  Readiness  Evaluation  Function 

The  Readiness  Evaluation  function  like  the  Performance  Evaluation  function  acts 
primarily  as  a  supplementary  element  to  the  Cost-Production  function.  The  Readiness 
Evaluation  function  can  perform  in  two  distinct  ways.  The  first  way  of  functioning  is 
represented  in  Figure  3  by  three  elements:  Requirements,  Resources,  and  Readiness. 
These  three  elements  provide  for  a  continuous  evaluation  of  how  well  the  prescriptive 
readiness  requirements  have  been  met--have  all  the  individuals  and  functional  units 
completed  the  required  training  courses  and  events.  The  second  way  of  functioning  uses 
all  the  elements  in  Figure  3  and  provides  a  possible  linkage  between  training  and  empirical 
readiness  evaluations. 

This  third  function  of  training  system  operation  looks  at  the  quantitative  and 
qualitative  effects  of  training  on  combat  readiness.  The  criterion  of  effectiveness  can  range 
from  a  certification  that  the  approved  training  requirements  have  been  met  to  an  actual 
measure  of  combat  relevant  performance.  The  outcomes  from  these  operations  are  capable 
of  having  very  significant  impact  upon  the  operation  of  the  training  system.  However, 
once  a  problem  has  been  satisfactorily  resolved,  the  training  system  goes  back  to  the 
baseline  operating  condition  represented  by  the  Cost-Production  function. 

C.  DISCUSSION  AND  CONCLUSIONS 

The  TMM  provides  a  conceptual  model  of  the  major  interactions  between  the 
macro-level  training  manager  and  the  training  system.  Based  upon  a  small  set  of 
assumptions  about  the  function  and  goals  of  the  macro-level  training  manager  and  the 
nature  of  training,  the  TMM  provides  a  broad  perspective  of  the  types  of  data  that  are  most 
likely  to  be  required  for  the  solution  of  training  policy  questions. 

The  TMM  provides  data  on  three  aspects  of  training:  Cost-Production,  Performance 
Effectiveness,  and  Readiness  Evaluation.  The  Cost-Production  function  is  the  normal 
operating  condition.  Here,  the  macro-level  training  manager  is  primarily  concerned  with 
costs,  the  effectiveness  of  the  training  system,  and  ways  to  improve  the  cost-effectiveness 
of  the  system.  If  the  effectiveness  or  cost-effectiveness  of  a  training  effort  becomes 
questionable,  then  the  Performance  Effectiveness  of  the  training  system  becomes  the 
dominant  concern.  Here,  performance  data,  primarily  of  individuals  or  groups  in  training 
or  recently  graduated  from  training,  become  the  macro-level  training  manager’s  primary 
area  of  concern.  If  the  contribution  of  training  to  combat  readiness  is  questionable,  then  the 
Readiness  Evaluation  aspect  of  training  becomes  dominant.  In  this  case,  the  focus  of  the 
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manager's  interest  may  be  either  the  adequacy  of  the  training  system  in  providing  all  the 
courses  and  unit  training  events  required  for  combat  readiness  or  else  how  well  the  courses 
and  unit  training  events  contributed  to  some  measure  of  combat  effectiveness.  When 
questions  concerning  Performance  Effectiveness  or  Readiness  Evaluation  have  been 
answered,  the  control  function  reverts  back  to  the  Cost-Production  aspect  of  training.  All 
three  functions  can  coexist  depending  on  what  aspects  of  training  are  dominant  at  any  time. 
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IV.  COST-PRODUCTION  FUNCTIONS  IN  TRAINING 


In  its  simplest  form,  the  Cost-Production  function  in  the  TMM  counts  the  number 
of  personnel  trained  and  compares  these  figures  with  the  resources  consumed  and  with  the 
number  of  trained  personnel  required.  More  sophisticated  versions  of  this  aspect  of  the 
training  system  might  consider  the  training  systems  as  being  directly  analogous  to  hardware 
production  lines.  A  training  program  can  be  viewed  as  having  start-up  costs,  and  fixed  and 
variable  costs  based  on  production  rates.  If  training  production  systems  are  linked  directly 
to  the  technical  and  organizational  systems  that  they  support,  then  the  training  data  bases 
can  be  used  to  evaluate  the  probable  consequences  of  policy  decisions. 

From  the  training  manager's  point  of  view,  the  Cost-Production  function  can 
provide  the  most  policy  relevant  information  about  the  training  system  of  any  of  the  major 
functions  if  the  following  two  assumptions  are  acceptable: 

1 .  That  training  courses  are  effective  in  conveying  the  necessary  knowledge  and 
skills  to  the  personnel  involved. 

2.  That  trained  personnel  advance  the  state  of  readiness  and  potential  combat 
effectiveness  of  the  armed  forces. 

These  assumptions  are  generally  acceptable  and  training  managers  normally  operate 
within  a  Cost-Production  frame  of  reference.  Consequently,  their  greatest  need  will  be  for 
training  data  base  information  that  assists  them  in  providing  the  required  number  of  trained 
personnel,  when  needed,  and  for  a  minimum  cost.  The  Cost-Production  function  in 
training  is  presented  in  Figure  4. 

Data  for  the  Cost-Production  function  cluster  into  three  categories  which  are 
reviewed  next:  level  of  effort,  training  capabilities  and  capacities,  and  training  proficiency. 
The  level  of  effort  category  contains  examples  of  the  kinds  of  information  used  to  assess 
levels  of  effort  in  training  such  as  student  training  loads  and  inventory  projections  for 
individual  training  and  flight  hours,  steaming  hours  and  battalion  training  days  for 
estimating  the  magnitude  of  collective  training.  The  training  capabilities  and  capacities 
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category  contains  the  descriptive,  trend,  and  functional  data  needed  by  the  training  manager 
to  properly  manage  and  develop  the  training  systems’  physical  and  personnel  resources. 
The  training  proficiency  category  contains  measure  of  how  well  the  training  system  has 
responded  to  approved  training  requirements. 

A.  TRAINING  SYSTEM  LEVEL  OF  EFFORT 

1 .  Individual  Training 

According  to  the  Department  of  Defense  Force  Readiness  Report  for  FY  1984 
(DoD,  1983),  the  level  of  effort  of  individual  training  can  be  monitored  through  the  use  of 
two  indices: 

1 .  Student  Training  Loads  -  indicate  the  total  person-years  of  individual  training 
contained  in  Service  budgets. 

2.  Inventory  Projections  -  indicate  the  effect  of  projected  training  loads  in 
reducing  occupational  imbalances. 

In  combination,  these  indices  provide  information  on  the  magnitude  of  the  Services' 
individual  training  plans  and  on  whether  these  plans  provide  enough  people  trained  in 
needed  skills. 

2.  Collective  Training 

The  level  of  effort  of  collective  training  can  be  monitored  through  a  number  of 
different  indices.  Flying  hours,  steaming  days,  and  battalion  training  days  are  the 
measures  used  most  frequently  to  gauge  the  approximate  level  of  effort  involved  in  team 
training. 


a.  Air  Warfare  Training 

Flying  hours  represent  the  principal  index  of  aviation  training  activities.  The 
Department  of  Defense  Force  Readiness  Report  for  FY  1986  (DoD,  1985)  noted  that  the 
flying  hour  program  is  the  largest  and  most  significant  of  all  Air  Force  training  programs. 
Approximately  35  percent  of  the  Air  Force  Operations  and  Maintenance  (O&M)  budget 
supports  flying  operations.  It  should  also  be  noted  that  the  training  value  of  flight  hours 
varies  according  to  the  purpose  of  the  flight.  With  respect  to  the  Air  Force,  the  DoD  Force 
Readiness  Report  (1983)  stated  that  the  data  on  flying  hours  per  pilot  per  aircraft  does  not 
differentiate  between  readiness  and  non-readiness  related  flying:  that  is,  those  flying  hours 
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directly  associated  with  increasing  the  pilot/crew's  proficiency  to  perform  combat  missions 
versus  the  hours  spent  in  collateral  or  support  flying. 

In  discussing  the  Navy's  flight  hour  program,  the  Department  of  Defense  Force 
Readiness  Report  for  FY  1986  (DoD,  1985)  presented  the  following  information: 

Primary  Mission  Requirements  (PMR)  constitute  those  hours  required  in 
TACAIR/ASW  flying  that  aircrews  must  accomplish  to  achieve  a  certain 
level  of  training.  It  also  encompasses  executing  fleet  operations  and  the 
performance  of  support  tasking.  The  Department  of  the  Navy  (DON)  goal 
is  88  percent  funding  of  PMR,  which  includes  two  percent  simulator  flying. 
Although  PMR  funding  percentage  has  declined  since  FY  1977,  the  DON 
budgeted  for  an  increased  percentage  funding  for  PMR  in  FY  1985  of 

86  percent  and  continued  this  upward  trend  with  a  FY  1986  funding  level  of 

87  percent 

The  difference  between  training  goals  and  budget  support  for  training  was  also 
noted  by  Cavalluzzo  (1985). 

Within  the  USN  air  community,  optempo  signifies  flying  hours  per  quarter. 

The  flying  hour  goal  was  stable  between  1977  and  1984.  Budgeted  flying 
hours  have  fallen  continuously,  however,  over  the  same  period.  For 
nondeployed  squadrons  alone,  the  numbers  are  even  lower  because 
deployed  squadrons  have  averaged  approximately  1 15  percent  of  goal,  and 
the  difference  must  be  made  up  by  the  nondeployed  community. 

The  flying  hour  is  also  used  as  the  basic  unit  for  planning  training  in  Army  aviation. 
According  to  the  Force  Readiness  Report  for  FY  1984  (DoD,  1983),  the  Active  Army 
Flying  Hour  Program  (FHP)  provides  the  minimum  number  of  flying  hours  needed  to 
support  individual  training  and  to  maintain  unit  level  proficiency  for  aviation  units  at  their 
programmed  manning  levels.  The  increase  in  flying  hours  from  year  to  year  is  related  not 
to  aircraft  inventory  but  to  aviator  inventory,  since  the  Army  FHP  is  based  on  the  number 
of  hours  required  to  attain  and  maintain  both  individual  and  unit  proficiency. 

A  major  problem  for  the  training  manager  is  the  task  of  estimating  the  impact  of 
partial  (i.e.,  "insufficient")  funding  on  training  readiness.  One  of  the  questions  that  needs 
to  be  addressed  is  whether  the  data  bases  will  enable  the  training  manager  to  predict  the 
relationship  between  various  amounts  of  partial  funding  and  quality  of  unit  performance  or 
readiness  level.  This  is  a  complex  question  that  would  require  the  use  of  two  of  more  of 
the  TMM’s  major  functions.  In  the  Cost-Production  function,  the  training  manager  should 
be  able  to  use  the  data  base  to  document  the  relationship  between  decreased  funding  and 
decreases  in  the  amount  of  training  time.  As  a  next  step,  either  the  Performance 
Effectiveness  or  the  Readiness  Evaluation  functions  would  be  invoked  to  document  the 
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relationship  between  decreased  training  time  and  measurable  decreases  in  indices  of 
performance  effectiveness.  However,  from  the  Cost-Production  perspective,  the  training 
manager  is  primarily  interested  in  how  well  the  prescriptive  requirements  are  being  met,  the 
cost  savings  from  any  new  instructional  techniques  or  technology  insertions,  and  whether 
conditions  are  improving  or  deteriorating. 

b .  Sea  Warfare  Training 

Steaming  hours  provide  a  measure  of  some  aspects  of  the  amount  of  crew  and  unit 
training  for  sea  warfare.  Like  flight  hours,  steaming  hours  represent  a  measure  of  training 
opportunity.  However,  there  are  no  production  equations  that  convert  steaming  hours  to 
either  training  readiness  or  unit  proficiency.  Cavalluzzo  (1985)  provided  some  insight 
concerning  the  ambiguousness  of  the  relationship  between  the  use  of  resources  and  its 
results  in  training. 

The  primary  mission  of  the  nondeployed  fleets  of  the  U.S.  Navy  (USN)  is 
to  prepare  personnel  for  deployment  On-the-job  training  while  underway 
allows  a  crew  to  work  as  an  integrated  unit  in  a  hands-on  setting.  But  it  is 
very  expensive,  and  until  now,  no  one  from  the  analytical  community  has 
tried  to  measure  the  readiness  payoff  of  the  expenditure.  Thus,  it  is  not 
possible  to  say  with  any  degree  of  precision  what  would  be  gained  by 
stepping  up  nondeployed  activities,  or  conversely,  what  would  be  lost  by 
cutting  back.  Although  more  work  is  needed,  preliminary  estimates  of  the 
association  between  nondeployed  activity  and  training  readiness,  based  on  a 
variety  of  measures,  are  positive. 

c.  Ground  Warfare  Training 

The  indices  of  levels  of  effort  of  ground  warfare  training  seem  to  be  more  variable 
and  more  subject  to  interpretation  than  the  training  resource  measures  used  in  the  other 
warfare  categories.  The  battalion  training  day  (BTD)  and  the  battalion  field  training  day 
(BFTD)  are  the  most  frequently  used  indices  of  level  of  effort  for  ground  warfare  training. 
The  DoD  Force  Readiness  Report  for  FY  1986  (DoD,  1985)  indicated  that  the  level  of 
effort  for  Army  collective  training  had  remained  relatively  stable  during  the  1983  to  1986 
time  frame.  "The  average  BTDs  per  battalion  remains  virtually  steady  at  about  162-164  per 
year." 

The  BTD  is  used  as  a  reporting  unit  but  not  as  a  planning  unit.  According  to  the 
Department  of  Defense  Force  Readiness  Report  for  FY  1984  (DoD,  1983),  Army  training 
is  not  managed  by  BTD  but  by  training  events,  which  are  planned  by  units  and  with  funds 
that  are  budgeted  well  in  advance  of  the  specific  fiscal  year.  BTD's  provide  a  general  index 
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of  the  amount  of  time  devoted  to  unit  training,  but  do  not  measure  the  contribution  of 
training  to  readiness.  Other  factors  such  as  availability  of  ranges,  training  facilities,  spare 
parts,  personnel  and  leader  fill/quality,  or  availability  of  fuel  are  significant  factors  that 
impact  the  amount  of  training  that  can  occur. 

The  Marine  Corps  uses  the  battalion  field  training  day  (BFTD)  as  a  reporting  unit. 
According  to  the  DoD  Force  Readiness  Report  for  FY  1984  (DoD,  1983), 

BFTD’s  are  developed  by  unit  commanders  to  quantify  collective  training 
conducted  in  the  unit  environment.  Planned  BFTD's  are  resource- 
constrained,  and  include  unit  deployments  as  well  as  local  training 
programs.  For  Marine  Corps  ground  force  units  BFTD's  include  that 
training  which: 

Is  designed  to  improve  the  unit's  mission  capability 
~  Is  at  least  eight  hours  in  duration 

Is  conducted  in  the  field  and/or  from  amphibious  shipping 
-  Involves  the  majority  of  the  unit's  effective  strength. 

The  use  of  situationally  unique  measures  of  training  levels  of  effort  significantly 
complicate  the  training  manager's  task.  Attempts  to  use  measures  that  cannot  be  combined 
and  which  are  loosely  related  to  training  goals  are  bound  to  produce  results  that  are  limited 
in  their  contributions  to  policymaking  functions. 

B .  TRAINING  CAPABILITIES  AND  CAPACITIES 

The  management,  development  and  structuring  of  the  capabilities  and  capacities  of 
the  training  system  represent  the  macro-level  training  manager's  unique  sphere  of 
influence.  As  such  it  may  be  the  single  most  important  area  for  the  formulation  of  training 
policy.  Many  of  the  major  parameters  influencing  the  operation  of  the  training  system 
(e.g.,  numbers  of  personnel  to  be  trained,  the  quality  of  the  personnel,  and  even  the  skills 
to  be  mastered)  are  externally  imposed  upon  the  training  system  and  its  managers.  As  with 
the  manager  of  a  manufacturing,  production,  and  distribution  system,  a  training  manager 
needs  to  know  a  great  deal  about  production  capabilities  and  capacities  in  order  to  make 
informed  cost-effective  decisions. 

Descriptive,  trend,  and  functional  information  are  needed  in  order  to  manage  the 
present  and  to  plan  for  the  future.  The  required  descriptive  information  includes:  budgets 
and  how  the  budgets  are  allocated  to  different  types  of  training;  training  capabilities; 
descriptions  of  the  kinds,  amounts,  and  the  unit  costs  of  the  training  accomplished;  and 
summaries  of  the  training  capabilities  including  their  capacities,  levels  of  utilization, 
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locations  and  physical  condition.  The  trend  information  should  be  able  to  provide  census 
types  of  information  so  that  the  training  manager  can  know  what  the  systems  capabilities 
used  to  be,  what  they  are  currently,  and  what  they  are  likely  to  be  in  the  future  (with  and 
without  anticipated  management  interventions).  The  functional  information  would  include 
formulas  and  descriptions  of  how  different  factors  influence  the  cost-effectiveness  of 
training  (e.g.,  decreases  in  the  quality  of  personnel  should  increase  training  times  and 
costs,  increases  in  the  use  of  computer-based  instruction  should  decrease  training  times  and 
possibly  costs  by  some  anticipated  amount,  and  a  knowledge  of  the  levels  of  personnel 
turbulence  and  rates  of  forgetting  could  (if  known)  be  used  to  establish  the  collective 
training  refresh  rates  required  to  maintain  the  desired  levels  of  unit  proficiency). 
Information  that  contributes  to  the  construction  and  evaluation  of  alternative  training 
scenarios  can  provide  the  training  manager  with  powerful  policymaking  tools. 

The  macro-level  training  manager  also  needs  to  be  able  to  aggregate  and  simplify 
the  capabilities  and  capacities  information.  A  training  system  with  a  worldwide  distribution 
of  assets  which  trains  people  in  hundreds  of  occupations  through  the  use  of  thousands  of 
courses  is  too  complex  for  any  individual  to  fully  comprehend;  hence,  the  need  for 
aggregation  and  simplification.  As  a  purely  imaginary  hypothetical  example,  giving  the 
training  manager  a  computer  listing  of  all  the  weapons  firing  ranges  in  DoD  with  then- 
locations  and  capabilities  would  be  of  minimal,  if  any,  value.  However,  if  all  of  the 
appropriate  range  data  were  summarized,  it  might  reveal  that  some  2,000  ranges  are 
currently  available  and  that  current  training  needs  require  the  use  of  1,500  ranges;  but  that 
due  to  environmental  problems,  1,600  ranges  would  go  out  of  operation  in  the  next  five 
years  and  that  most  of  the  remaining  ranges  would  be  in  Alaska  and  several  foreign 
countries.  This  sort  of  summarization  of  capabilities,  needs  and  trends  would  clearly 
provide  the  basis  for  formulating  training  policy. 

C.  TRAINING  SYSTEM  PROFICIENCY 

This  section  will  present  some  examples  of  data  relating  to  the  quantitative  aspects 
of  training  system  proficiency.  These  data  include  include  examples  from  both  individual 
and  collective  training.  The  measures  of  system  proficiency  are  indices  of  how  the  training 
system  has  responded  to  the  prescriptive  quantities,  types,  and  levels  of  individual  and 
collective  training  deemed  necessary  for  the  desired  state  of  national  readiness. 


1 .  Individual  Training 

The  sequential  training  milestones  that  need  to  be  satisfied  to  become  a  Fully 
Combat  Qualified  Marine  aircrewman  provide  a  good  example  of  a  quantitative  relationship 
between  the  specified  training  program  and  the  desired  training  proficiency  goal.  The  DoD 
Force  Readiness  Report  for  FY  1984  (DoD,  1983)  provides  an  example  of  prescriptive 
levels  of  training: 

The  following  training  sequence  generally  applies  to  produce  a  combat 
qualified  Marine  aircrewman:  Marine  flight  students  are  trained  in  Navy 
training  squadrons  to  the  point  where  they  are  formally  designated  as  Naval 
Aviators  or  Naval  Flight  Officers  (25%  ready).  They  then  proceed  to 
Marine  training  squadrons  where  they  receive  training  in  a  specific  type 
aircraft  to  become  Combat  Capable  (60%  ready).  The  new  aviator  is  then 
assigned  to  an  operational  squadron,  which  trains  him  to  become  Combat 
Ready  (70%  ready),  Combat  Qualified  (85%  ready),  and  Fully  Combat 
Qualified  (100%  ready). 

The  training  manager  needs  to  know  how  many  should  have  been  trained  to  each  level, 
how  many  were  trained  to  each  level,  and  at  what  cost.  In  order  to  determine  how  well  the 
system  is  performing,  it  is  also  necessary  to  have  data  from  past  years  to  determine  how 
well  system  production  and  costs  compare  with  previous  efforts  and  to  determine  whether 
there  are  any  observable  trends. 

2.  Collective  Training 

Prescriptive  training  readiness  is  a  joint  product  of  the  number  of  personnel 
assigned  to  a  unit,  the  number  and  types  of  individual  training  courses  that  they  have 
completed,  and  the  collective  training  exercises  that  the  unit  has  completed.  The  DoD  Force 
Readiness  Report  for  FY  1984  (DoD,  1983)  indicated  that  with  respect  to  individual 
training, 

The  formal  school  network  must  qualify  individuals  in  occupations 
necessary  to  staff  programmed  manning  vacancies.  This  training  will  affect 
personnel  readiness,  which  is  reported  by  matching  on-board  trained 
personnel  to  the  unit’s  programmed  structure. 

The  interrelationship  between  training  readiness  and  other  readiness  variables  was 
emphasized  in  the  same  publication  (DoD,  1983)  by  the  statement 

To  a  large  degree  training  readiness  is  a  dependent  variable,  very  much 
influenced  by  a  unit's  personnel  and  equipment  status. 


28 


Several  studies  have  reported  that  the  amount  of  training  influences  training 
readiness.  Horowitz  and  Angier  (1985)  reported  that  the  "quantity  of  formal  Navy  training 
was  a  consistent  predictor  of  ship  readiness."  Cavalluzzo  (1985)  reported  that  a  "1-day- 
per-quarter  rise  in  training  is  associated  with  a  2.26  percent  rise  in  the  number  of  ships  that 
are  combat  ready  upon  deployment." 

D.  DISCUSSION  AND  CONCLUSIONS 

In  order  for  a  data  base  to  support  the  Cost-Production  aspect  of  making  training 
policy,  it  should  be  able  to  provide  information  regarding  the  allocation  of  funds  for 
training,  cost  of  training,  levels  of  effort,  and  measures  of  system  proficiency.  The  present 
analysis  assumes  that  data  describing  the  funds  allocated  to  training  and  training  costs  are 
available;  however,  whether  they  are  the  right  kind  or  are  in  appropriate  formats  are 
questions  that  need  to  be  addressed  in  a  future  analysis.  Generally  accepted  measures  of 
levels  of  effort  are  available  in  the  form  of  measures  such  as  student  training  loads  and 
inventory  projections  for  individual  training  and  flying  hours,  steaming  hours,  and 
battalion  training  days  for  collective  training.  Some  training  system  proficiency  measures 
such  as  training  program  completions,  productivity  estimates,  and  training  readiness 
reports  are  either  available  or  at  least  obtainable. 

Second  only  to  the  adequacy  of  training  production  is  the  status  and  capability  of 
the  training  system.  A  significant  proportion  of  training  dollars  is  invested  in  the  facilities, 
equipment,  and  organizations  needed  to  conduct  training.  The  macro-level  training 
manager  needs  to  know  what  the  training  capabilities  are,  where  they  are  located,  what  their 
capacities  are,  their  degree  of  utilization,  their  condition  and  whether  they  are  improving  or 
deteriorating.  Since  many  policy  decisions  are  made  with  respect  to  these  training 
capabilities,  it  is  essential  that  the  data  banks  be  capable  of  providing  both  current 
descriptive  summaries  and  historical  trends  regarding  these  training  capabilities.  Catalogs 
of  training  capabilities  and  annual  summaries  of  training  production  accomplishments  are 
important  aids  to  the  policymaking  process. 

In  a  generic  sense,  we  have  identified  the  types  of  data  needed  to  support  cost- 
effectiveness  analyses  of  the  Cost-Production  aspect  of  training  operations.  Examples  of 
the  required  data  types  have  been  presented.  These  data  either  are  or  could  be  routinely 
acquired  for  use  within  training  data  bases.  Based  upon  the  TMM,  the  development  of  data 
bases  to  support  management  requirements  within  the  Cost-Production  function  should 
receive  the  highest  priority  because:  (1)  this  is  the  normal  frame  of  reference  for  a  macro 
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level  training  manager,  and  (2)  the  data  appear  to  be  of  the  sort  that  could  be  collected 
routinely  for  use  in  training  data  bases.  Further  work  is  needed  to  determine  the  extent  to 
which  the  required  data  exist  in  current  training  data  bases  and  whether  they  are  the  proper 
form  to  be  used  to  answer  specific  training  policy  questions. 

E.  DATA  BASE  OPTIONS  AND  STRATEGIES 

The  development  of  a  data  base  to  support  the  Cost-Production  aspect  of  training 
system  operation  should  be  the  macro-level  training  manager's  first  priority.  The  short¬ 
term  best  option  would  appear  to  be  one  in  which  fairly  simple  descriptive  data  bases  were 
established.  These  data  bases  should  be  capable  of  accurately  tracking  and  summarizing 
budget  allocations  for  training,  training  costs,  student  loads  and  inventory  projections. 
Additional  data  bases  should  be  capable  of  providing  descriptive  summaries  of  training 
facilities  and  equipment— including  their  capabilities,  capacities,  location  and  level  of 
utilization.  Efforts  should  be  made  to  be  able  to  list,  summarize,  and  simplify  the  data  in 
periodic  and  on-demand  reports  to  aid  in  the  development  of  training  policy. 

On  an  intermediate  term  basis  it  should  be  possible  to  develop  data  on  trends  by 
incrementing  the  data  files  on  an  annual  basis.  This  trend  data  would  assist  the  training 
manager  in  specifying  what  the  training  accomplishments  and  capabilities  were,  what  they 
are  now,  and  what  they  are  likely  to  become.  This  is  the  kind  of  training  management 
information  that  would  assist  DoD  and  Service  level  training  managers  in  responding  to 
Congressional  requests  for  information  and  program  justification  by  relating  past  levels  of 
funding  support  to  current  accomplishments  and  relating  current  and  anticipated  problems 
to  projected  needs  for  support. 

Probably  the  most  important  long-term  strategy  would  be  one  which  planned  the 
development  of  the  data  bases  as  a  means  of  developing  and  verifying  key  functional 
relationships.  In  both  the  development  and  justification  of  training  policy,  the  training 
manager  needs  to  know  or  to  be  able  to  project  the  effects  of  the  training  policy  on  the  cost- 
effectiveness  of  training.  The  ideal  goal  would  be  to  be  able  to  simulate  the  operation  of  the 
total  training  system  because  of  its  obvious  utility  for  analytical  purposes.  However,  there 
are  many  levels  of  sophistication  between  where  we  are  now  and  this  kind  of  an  ideal  goal. 
Considerable  benefits  could  be  accrued  by  systematically  collecting  from  the  training  data 
base,  special  data  bases,  and  the  research  literature  information  on  the  functional 
relationships  between  key  factors  influencing  the  operation  of  the  training  system  and  their 
effects  on  costs,  system  productivity,  and  the  cost-effectiveness  of  training.  This 
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information  could  be  summarized  in  a  dictionary  of  training  management  information.  The 
known  information  could  assist  in  answering  "What  if’  questions  and  in  identifying  gaps  in 
needed  management  information.  These  identified  information  gaps  could  be  used  to  guide 
the  course  of  research  and  future  data  base  development 


V.  PERFORMANCE  EFFECTIVENESS  FUNCTIONS  IN 

TRAINING 


Performance  quality/achievement  data,  particularly  when  the  performance  is 
significantly  below  established  standards  or  has  been  reported  as  unsatisfactory  by  a  blue 
ribbon  panel,  can  lead  the  manager  to  a  consideration  of  the  Performance  Effectiveness 
function  of  training  operations.  The  results  of  performance  effectiveness  evaluations  can 
produce  significant  and  lasting  changes  in  the  training  system.  However,  once  the  changes 
have  been  programmed,  budgeted  and  implemented,  interest  reverts  to  the  Cost-Production 
aspect  of  training  operation.  Since  the  training  is  now  assumed  to  remain  satisfactory,  the 
performance  effectiveness  data  cease  to  be  of  major  concern  at  the  policy  level. 

The  information  in  this  chapter  will  be  presented  in  two  sections.  The  first  section 
will  discuss  the  characteristics  of  the  Performance  Effectiveness  function  of  the  Training 
Management  Model  (TMM).  The  second  section  will  present  and  discuss  examples  of  the 
kinds  of  data  needed  to  assess  the  effectiveness  of  training.  These  data  were  derived  from 
individual  and  collective  training.  The  focus  of  the  data  is  either  on  the  quality  of 
performance  during  training  or  on  some  measure  of  operationally  relevant  performance. 
When  these  measures  are  aggregated  and  when  they  can  be  related  to  training  techniques, 
the  amount  of  resources  used,  or  the  intensity  of  the  training  activity  required  to  achieve 
different  levels  of  training  results,  they  become  significant  to  the  macro  operation  of  the 
training  system. 

A.  DESCRIPTION  OF  THE  PERFORMANCE  EFFECTIVENESS 
FUNCTION  IN  TRAINING 

Training  is  an  acknowledged  obligation  of  all  military  organizations  and  is  usually 
assumed  to  be  at  least  minimally  effective  unless  proven  otherwise.  (Since  training  is 
under  the  direction  of  personnel  experienced  in  the  specialties  being  taught,  this  is  usually  a 
reasonable  assumption.)  Perceptions  of  training  inadequacies  may  serve  to  generate 
sufficient  interest  in  a  problem  area  so  that  a  special  study  is  conducted  and  changes  to  the 
training  system  are  proposed  and  approved.  In  some  situations  the  reported  observations 


of  the  quality  of  trained  performance  may  involve  the  locus  of  responsibility  for  training  as 
well  as  the  quality  of  training.  For  example,  perceptions  of  a  problem  in  Navy  maintenance 
training  were  presented  by  the  Commander  in  Chief,  U.S.  Atlantic  Fleet  (CINCLANTFLT) 
in  a  message  (1981)  to  the  Chief  of  Navy  Technical  Training  (CNTECHTRA): 

Today's  apprentice-technicians  are  generally  incapable  of  carrying  out  the 
basic  tasks  related  to  equipment  maintenance.  As  a  result,  the  senior  petty 
officer  is  required  to  fulfill  these  basic  tasks  and  outside  assistance  must  be 
called  in  to  accomplish  journeyman  petty  officer  tasks,  (cited  in  Keeler  & 

Guynn,  1986). 

The  message  clearly  indicates  that  a  higher  level  of  training  is  needed  before  the  technicians 
are  sent  to  the  fleet  Later,  the  Navy  decided  to  transfer  a  greater  portion  of  the  training 
responsibility  to  the  fleet.  In  1986,  Burlage  reported  the  Navy's  decision  that 

Schoolhouse  training  for  sailors  entering  18  support  ratings  will  be  reduced 
an  average  of  25  percent  by  October  as  part  of  the  Navy's  newly  announced 
plans  to  cut  formal  instruction  and  shift  more  of  its  responsibility  to  the 
fleet. 

It  is  significant  that  neither  of  these  judgements  regarding  either  the  adequacy  or  the  agent 
responsible  for  training  directly  address  the  skill  levels  to  be  attained  through  formal  initial 
skill  training.  Better  representation  and  quantification  of  apprentice  and  journeyman  skill 
levels  would  add  significantly  to  the  precision  and  justifiability  of  training  management 
decisions. 

Figure  5  represents  the  Performance  Effectiveness  function  of  training  system 
operation;  the  performance  evaluation  element  is  labeled  Performance  Evaluation/Quality 
Control.  The  block  diagram  illustrates  several  characteristics  of  the  Performance 
Effectiveness  aspect  of  training.  First,  it  is  a  supplement  to  the  Cost-Production  aspect  of 
training.  Second,  while  it  can  influence  either  individual  and  collective  training  and/or  the 
requirements  for  each,  performance  data  are  used  principally  to  evaluate  the  adequacy  of 
student  progress  in  individual  training.  If  student  evaluation  scores  are  above  the  minimum 
acceptable  level,  the  students  are  retained  for  the  next  phase  of  training  or  are  graduated 
from  the  course.  If  the  student  scores  are  below  the  minimum  acceptable  level,  the  students 
are  either  recycled  through  the  failed  training  segment  or  they  are  dropped  from  the  course. 
This  process  is  analogous  to  the  use  of  multiple  product  inspections  on  an  assembly  line. 
The  primary  function  of  student  evaluations  is  one  of  quality  control  to  assure  that  no 
technically  unsatisfactory  students  are  graduated  from  the  training  program.  (At  this  point 
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it  may  be  worth  noting  that  although  a  great  deal  of  learning  occurs  during  training,  the 
function  of  training  is  more  directly  analogous  to  the  function  of  the  shaping  phase  of  a 
classical  instrumental  learning  paradigm.  Like  shaping,  training  consists  of  a  number  of 
preliminary  familiarization  and  learning  experiences  that  ensure  that  the  desired  behaviors 
are  likely  to  occur.  The  types  of  performance  data,  decreasing  error  rates  or  increasing 
speed  over  time  that  are  the  essence  of  traditional  learning  curves  do  not  occur  until  the 
student  completes  training  and  begins  to  apply  his  or  her  skills  on  a  regular  basis  [examples 
of  this  type  of  on-the-job  learning  curve  can  be  found  in  Quester  and  Marcus  (1985)]. 
However,  the  extensiveness  or  the  quality  of  the  training  determines  the  magnitude  of  the 
first  point  on  the  curve  and  probably  the  slope  of  the  curve  as  well;  i.e.,  more  fully  trained 
personnel  probably  demonstrate  higher  levels  of  initial  performance  on  the  job  and  improve 
at  a  faster  rate  than  less  effectively  trained  personnel  (Gibson  and  Orlansky,  1986).) 

Another  major  limitation  on  the  macro-level  use  of  performance  data  stems  from  the 
fact  that  they  are  usually  very  specialized  and  cannot  be  meaningfully  aggregated. 
Consequently,  performance  data  are  most  likely  to  serve  as  a  quality  control  element  for  a 
particular  training  course.  In  this  capacity  its  existence  and  operation  are  generally 
irrelevant  to  the  needs  and  interests  of  the  macro-level  training  manager.  On  those 
occasions  when  performance  data  reach  the  level  of  visibility  needed  to  influence 
requirements,  it  is  likely  to  be  in  a  major  way  which  will  require  a  long-term  change  in 
resource  allocation  caused  by  the  addition  or  deletion  of  training  programs.  Any  time  that 
the  quality  of  training  is  seriously  questioned  or  any  time  that  the  macro-level  manager  feels 
that  proposed  training  changes  may  affect  training  quality,  it  will  be  necessary  to  have 
access  to  the  appropriate  performance  effectiveness  data. 

B .  PERFORMANCE  EFFECTIVENESS  DATA 

1.  Individual  Training 

a.  Aviation  Training 

Flying  hours  have  been  found  to  be  related  to  operational  performance  in  the  forms 
of  boarding  rates  (i.e.,  percentage  of  successful  carrier  landing  attempts),  Operational 
Readiness  Evaluation  (ORE)  grades  and  bombing  accuracy.  Cavalluzzo  (1985)  reported 
that  as  flying  hours  declined,  there  is  a  substantial  increase  in  the  dispersion  of  boarding 
rates.  At  higher  flying-hour  rates,  average  boarding  rates  were  higher  and  clustered  more 
closely  to  the  average  (see  Table  1).  Cavalluzzo’s  analysis  of  the  ORE  grades  closely 
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paralleled  the  results  of  the  boarding  rate  analysis.  Higher  squadron  ORE  grades  were 
associated  with  higher  average  monthly  flying  hours  (see  Table  2).  Cavalluzzo  noted  that 
the  relationship  between  flying  hours  and  ORE  scores  also  held  true  for  interfleet 
comparisons. 

Interfleet  comparisons  yielded  an  equally  dramatic  difference.  Squadrons  in 
the  Pacific  Fleet  average  less  than  80  percent  as  many  monthly  pre-ORE 
flying  hours  as  those  in  the  Atlantic  Fleet  Only  39  percent  of  Pacific  Fleet 
squadrons  received  scores  in  the  top  two  ORE  categories,  compared  to 
63  percent  for  the  Atlantic  Fleet 

Table  1.  Average  Boarding  Rates  During  Operational  Readiness  Evaluation 

(ORE)® 


Flying  Hours 

Per  Squadron 

Per  Month 

Average 

Boarding 

Rate 

Minimum 

Boarding 

Rate 

Lower  25th 
Percentile 
Boarding  Rate 

Under  360 

88.5 

70.0 

85.0 

360  -  439 

90.1 

70.5 

87.5 

440  -  519 

93.2 

84.0 

91.2 

Over  519 

94.1 

89.3 

n/a 

®  Cavalluzzo.  1985,  p.  372. 

Table  2.  ORE  Grades  and  Monthly  Flying  Hours® 


Average  Monthly 

ORE  Grade 

Flying  Hours 

Excellent 

356 

High  Excellent 

384 

Low  Outstanding 

421 

Outstanding 

487 

a  Cavalluzzo,  1985,  p.  372. 


In  the  same  report,  she  also  noted  that  a  1 -percent  increase  in  flying  hours  devoted 
to  bombing  practice  was  associated  with  a  1/2-percent  reduction  in  average  miss  distance. 
In  the  case  of  bombing  accuracy,  the  relationship  between  practice  and  accuracy  is  not  a 
simple  linear  relationship.  Cavalluzzo  (1985)  reported 

...The  United  States  Air  Force  has  some  preliminary  work  that  indicates  that 
bombing  accuracy  is  not  something  which  improves  gradually  with  practice, 
but  is  instead  susceptible  to  sudden  improvement  at  a  later  stage,  that  is,  at 
"journeyman"  rather  than  "apprentice"  level,  when  trainees  suddenly  pick 
up  the  knack.  If  this  is  the  case,  it  leads  one  to  believe  that  as  much  as  two 
years  lead  time  may  be  required  in  order  to  ensure  a  high  degree  of 
accuracy. 

These  findings  suggest  that  the  amount  of  training  and  the  operational  results  for  a 
cohort  of  trainees  may  need  to  be  tracked  for  at  least  two  years  before  the  effects  of 
program  increments  or  decrements  can  be  detected.  The  notion  of  step  increases  in  the 
relationship  between  resource  usage  and  the  effectiveness  of  the  targeted  performance 
would  be  important  in  justifying  the  level  and  duration  of  support  for  some  training 
activities.  While  the  specific  relationships  would  have  to  be  documented  at  a  less  than 
macro  level,  the  verified  relationships  could  be  of  value  in  the  formulation  of  macro-level 
training  policies. 

Some  useful  conceptual  work  and  research  has  been  done  to  relate  performance 
quality /achievement  to  the  cost-effectiveness  of  aviation  trainers.  Orlansky  (1985) 
reviewed  a  series  of  studies  which  developed  a  methodology  for  assessing  the  cost- 
effectiveness  of  aircraft  simulators.  The  technique  compares  the  relative  costs  of  training  in 
simulators  and  in  aircraft  with  a  measure  of  the  relative  effectiveness  of  progressively 
increasing  amounts  of  simulator  training.  This  approach  combines  both  cost  and 
performance  quality/achievement  data  to  specify  the  cost-effective  range  for  simulator 
training.  In  effect,  the  use  of  simulator  training  is  cost-effective  until  the  incremental  gain 
in  training  benefits  from  simulators  relative  to  flying  is  less  than  the  ratio  of  simulator  to 
aircraft  training  costs. 

The  collection  and  application  of  this  type  of  specific  simulator  evaluation  data  has 
strong  implications  for  the  development  of  macro-level  training  management  policy.  Where 
simulators  are  an  optional  method  of  training,  this  technique  can  serve  as  a  model  of  the 
kinds  of  data  that  need  to  be  collected  and  the  way  that  they  can  be  used  to  formulate 
training  policy.  It  would  be  interesting  to  see  how  useful  this  approach  would  be  in 
evaluating  the  cost-effectiveness  of  other  simulator  applications  and  computer-based 
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instruction  (CBI).  In  principle,  it  would  apply  to  many  different  types  of  simulation  used 
as  an  optional  training  method  in  lieu  of  operational  equipment  and/or  the  duration  of  field 
training  exercises. 

b.  Ground  Force  Training 

Measurement  of  the  effects  of  individual  training  on  the  performance  effectiveness/ 
achievement  appears  to  be  more  difficult  for  Army  and  Marine  Corps  ground  forces  than  it 
is  for  aviation.  This  is  in  part  due  to  the  fact  that  the  individual  performances  occur  within 
a  complex  of  other  concurrent  activities.  Pellicci  (1985)  has  noted  that  "Individual  training 
is  a  continuous  process  but— because  it  is  conducted  concurrently  with  other  activities— it 
requires  few  dedicated  resources,  and  is  not  a  significant  cost  driver  in  non-aviation  units." 
This  concurrence  with  other  activities  complicates  the  measurement  problem  for  two 
reasons:  (1)  any  set  of  measures  will  be  confounded  by  the  concurrent  activities,  and 
(2)  no  single  measure  adequately  describes  the  total  set  of  group  activities. 

Some  attempts  to  assess  training  effectiveness  have  produced  mixed  results.  A 
report  on  the  use  of  battalion  ORE  scores  to  evaluate  the  effectiveness  of  training  for  the 
Hawk  missile  by  Tubbs,  Hansen,  Roberts,  Abel,  and  Wilson  (1985)  was  equivocal  in 
trying  to  determine  the  cause  of  the  consistently  low  ORE  scores  (see  Table  3). 

The  results  of  the  OREs  underscore  the  fact  that,  in  addition  to  the  standards 
(which  are  extracted  from  current  Technical  Manuals  (TMs))  perhaps  being 
too  strict,  training  and  maintenance  are  not  adequate  to  support  the  required 
unit  readiness.  While  crew  performance  is  low,  reflecting  the  state  of 
training,  the  equipment  performance  reflected  by  the  combined  score,  is 
even  worse. 

Clearly,  performance  measured  against  the  ORE  standards  was  not  very  good;  however, 
since  these  standards  were  judged  as  being  too  strict,  it  is  hard  to  evaluate  training 
effectiveness  since  small  changes  in  the  standards  might  result  in  large  changes  in  the  ORE 
performance  scores. 

Many  of  the  problems  associated  with  evaluating  training  effectiveness  may  be 
solved  through  computer  modeling  and  combat  simulations.  Neal  and  Paris  (1985) 
reported 

There  is  a  new  effort  at  the  U.S.  Army  TRADOC  Systems  Analysis  Activity 
(TRASANA)  to  use  computer  combat  simulation  models  to  evaluate  the 
operational  significance  of  training  performance  criteria  and  soldier 
performance  resulting  from  training  programs. 
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Table  3.  Battalion  ORE  Results8 


Crew  Performance 
(percent) 

Combined  Crew 
and  Equipment 
(percent) 

Number  of 
OREs 

Fire  Control 

Launcher 

^>"''''>*jGoal 

Battalion 

100 

100 

75 

1 

20 

50 

30 

10 

2 

10 

71 

0 

7 

3 

23 

66 

0 

3 

4 

52 

47 

43 

23 

5 

55 

42 

67 

24 

6 

42 

57 

29 

7 

Average 

35 

55 

28 

12 

a  Tubbs,  Hansen,  Roberts,  Abel  &  Wilson,  1985,  p.  182. 

The  availability  of  empirically-based  performance  standards  would  be  a  tremendous  asset 
for  training  managers.  The  ability  to  specify  the  impact  of  different  levels  of  training  on 
combat  effectiveness  would  give  the  macro-level  training  manager  greatly  increased 
leverage  in  justifying  and  defending  training  budgets.  If  nothing  else,  the  computer  models 
should  be  useful  in  identifying  the  measures  and  performance  relationships  that  should  be 
validated  through  the  collection  of  field  data. 

c.  Maintenance  Training 

A  large  proportion  of  the  Services’  individual  training  programs  are  devoted  to 
training  the  technicians  who  maintain  the  vast  array  of  equipment  used  by  the  military. 
Consequently,  the  assessment  of  the  cost-effectiveness  of  these  programs  is  important  to 
training  management. 
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Gibson  and  Orlansky  (1986)  reviewed  a  number  of  job  performance  measures  that 
have  been  used  to  evaluate  maintenance  training  effectiveness.  Several  measures  were 
identified  as  meriting  consideration  for  future  training  effectiveness  evaluations:  speed  of 
work  and  net  productivity  ratings.  Data  on  the  speed  of  work  completion  reported  in 
maintenance  data  banks  was  sensitive  to  differences  in  training  techniques.  For  example, 
studies  on  the  effects  of  additional  formal-type  training  (Field  Detachment  Training-FTD) 
for  F-16  technicians  indicated  that  those  who  had  such  training  performed  better  and 
seemed  to  benefit  more  from  their  work  experience  than  those  who  did  not  have  the 
additional  formal  training  [Johnson,  McConnell  &  Murdock  (1983)  and  McConnell  & 
Johnson  (1984)].  A  report  by  Quester  and  Marcus  (1985)  using  net  productivity  ratings, 
indicated  that  technicians  who  had  received  formal  A-school  training  were  more  productive 
on  the  job  than  those  who  had  received  OJT.  (The  use  of  a  net  productivity  rating  also 
provided  a  common  metric  for  doing  a  cost-effectiveness  analysis  of  the  comparative 
advantage  of  A-school  versus  OJT  for  a  number  of  technician  ratings.) 

Most  of  the  maintenance  job  performance  data  could  be  expected  to  serve  a  quality 
control  function  at  the  micro  levels  of  training  management  However,  the  development  of 
tools  to  objectively  select  training  methods  and  to  determine  the  cost-effectiveness  of 
training  also  have  obvious  implications  for  the  formulation  of  training  policy  at  the  macro 
level  of  training  management 

2.  Collective  Training 

Most  forms  of  collective  training  are  so  complex  that  they  have  previously  defied 
the  use  of  the  available  measurement  and  assessment  techniques.  Consequently,  few 
performance  effectiveness  data  have  been  gathered  and  the  available  data  have  not  had  a 
significant  effect  upon  training  requirements.  Fortunately,  the  future  for  collective  training 
evaluations  may  be  substantially  brighter.  Advances  in  the  form  of  instrumented  ranges, 
simulated  weapons  fire,  networked  simulators,  and  advanced  multi-element  combat 
simulations  promise  to  revolutionize  our  ability  to  measure  and  evaluate  collective  training. 
Training  managers  need  to  develop  plans  for  the  effective  use  of  these  improved  forms  of 
collective  training  evaluation. 

Rowland  (1985)  provides  an  innovative  example  of  how  collective  performance 
effectiveness  data  can  be  collected  and  used.  The  results  of  an  interactive  combat  training 
exercise  called  REALTRAIN  were  used  to  show  that  some  collective  training  activities  can 
be  extremely  effective. 
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From  Lanchester's  square  law— confirmed  by  analysis  of  simulated 
interactive  armoured  battles-the  exchange  ratio  will  be  inversely 
proportional  to  the  number  of  weapons  on  each  side  in  battles.  Thus  for 
defensive  battles  a  change  from  standard  troops  at  50%  proficiency  at  the 
beginning  of  REALTRAIN  to  95%  after  two  weeks  can  be  equated  to  the 
effect  of  a  95/50  or  1.9X  increase  in  force  size  without  such  extra 
training. ...Evidently  major  changes  in  capability  are  possible  if  practice  in 
interactive  combat  is  provided  at  platoon  level  or  above. 

Rowland's  evaluation  of  the  effects  of  collective  training  as  a  force  multiplier  would  seem 
to  have  real  value  in  estimating  the  cost-effectiveness  of  training.  If  this  type  of  data  could 
be  collected  on  a  larger  scale  and  for  a  significant  array  of  applications,  it  would  allow 
training  managers  to  develop  trade-offs  between  the  costs  of  training  and  the  costs  of 
increased  force  size  or  between  more  training  and  less  combat  fire  power.  This  would  be  a 
very  powerful  tool  for  the  training  manager. 

Currently,  however,  except  for  some  global  forms  of  expert  judgment  (e.g., 
Operational  Readiness  Evaluation  (ORE)),  neither  the  training  program  nor  the  trainees  are 
subjected  to  routine  evaluations.  Generally,  participation  in  what  is  called  "collective 
training"  is  taken  to  be  equivalent  to  being  trained;  i.e.,  the  collective  training  fulfills  a 
prescriptive  requirement  and  it  is  assumed  to  be  effective.  For  these  reasons  the  collection 
and  feedback  of  quality  control  information  assumes  the  probabilistic  character  of  an  event 
which  may  or  may  not  happen. 

The  scarcity  of  data  on  collective  training  effectiveness  can  be  attributed  to  two 
major  reasons:  (1)  Most  collective  training  events  are  complex  and  subject  to  uncontrolled 
environmental,  organizational  and  social  variables.  Consequently,  performance  measures 
are  difficult  to  obtain  and  even  more  difficult  to  interpret.  (2)  Many  training  organizations 
are  resistant  to  the  idea  of  performance  measurement  on  the  basis  that  the  measurement 
process  changes  the  significance  of  the  event  and  inhibits  learning.  The  flavor  of  an  anti¬ 
measurement  training  ethic  was  presented  by  Skinner  (1984)  in  his  description  of  the 
Air  Force's  "Red  Flag"  air  combat  training  program. 

"We  do  not  evaluate  people  at  Red  Flag.  We  don't  give  them  check  rides, 
and  we  don't  let  the  wings  have  Operational  Readiness  Inspections  at  Red 
Flag.  No  evaluations,"  says  the  Red  Flag  CO.  "It's  a  learning  program, 
and  we  don't  want  them  put  under  the  pressure  of  a  test  or  evaluation 
because,  invariably,  people  who  are  being  tested  do  not  learn. 

As  the  real-time  data  links  from  the  range  become  more  widespread  and 
sophisticated,  there  will  be  an  even  greater  tendency  for  outsiders, 
particularly  contractors  and  military  reformers,  to  keep  score'  at  Red  Flag." 
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Even  with  the  assumption  that  collective  training  always  accomplishes  its  training 
objectives,  a  major  outstanding  question  remains:  How  long  do  the  effects  of  the  collective 
training  last?  Answers  to  this  question  would  have  significant  influence  on  macro  level 
training  policy  decisions  in  terms  of  the  training  refresh  rates,  and  the  number  and  size  of 
the  training  facilities  and  materials  needed  to  maintain  minimum  acceptable  training  levels. 

C.  DISCUSSION  AND  CONCLUSIONS 

In  order  for  a  data  base  to  support  policymaking  efforts  on  the  basis  of  Performance 
Effectiveness  data  and  analyses,  the  data  should  be  able  to  provide  information  on  the  skill 
levels  achieved  through  initial  skill  training,  on-the-job  training,  and  post-initial-skills 
individual,  team  and  unit  training.  The  training  manager  should  be  able  to  relate  these 
achieved  skill  levels  to  the  skill  level  mixes  needed  for  effective  military  operations.  If  the 
skill  levels  can  be  adequately  quantified,  it  should  also  be  possible  to  specify  the  time  and 
resources  required  to  achieve  desired  skill  levels  for  specified  military  occupational 
specialties  and/or  for  particular  weapon  systems.  The  macro-level  manager  needs 
performance  effectiveness  data  when  the  effectiveness  of  a  training  program  is  under 
review.  Such  occasions  are  most  likely  to  occur  as  the  result  of  complaints  from  the 
operating  forces  or  as  the  result  of  changes  in  the  training  program.  The  training  changes 
may  be  due  to  the  introduction  of  new  training  techniques  or  technological  innovations 
intended  either  to  improve  training  quality  or  to  save  training  resources  while  preserving 
training  quality. 

If  "perfect"  performance  measures  were  available  to  the  training  manager,  they 
could  be  used  to  establish  training  goals  and  periodically  to  verify  training  results.  Once 
the  goals  were  established  or  the  results  were  satisfactorily  verified,  management  interest 
could  return  to  the  Cost-Production  function  of  training.  Except  when  the  effectiveness  of 
training  is  in  doubt,  the  macro-level  manager's  primary  concern  is  to  produce  the  necessary 
quantities  of  trained  personnel  for  the  least  cost 

While  the  macro-level  training  manager's  needs  for  performance  effectiveness  data 
are  limited,  the  micro-level  training  program  and  course  managers  have  a  continual  need  for 
this  kind  of  information.  Performance  effectiveness  data  provides  micro-level  managers 
with  their  most  important  quality  control  tool.  By  monitoring  how  well  students  perform 
during  training,  they  can  identify  those  who  can  progress  to  the  next  step  or  graduate,  those 
who  need  additional  training  and  those  who  should  be  dropped  from  the  program. 


The  performance  effectiveness  data  requirements  can  be  divided  into  two  categories: 
(1)  data  which  relate  the  performance  levels  achieved  through  formal  training  programs  to 
the  skill  levels  and  mixes  required  by  the  operating  forces,  and  (2)  data  which  relate  post- 
initial-skills  training  activities  such  as  flying  hours  to  maintaining  or  improving  skill  levels 
while  performing  in  the  operational  environment.  Test  scores  and  course  grades  satisfy 
part  of  the  first  requirement  and  they  provide  a  basis  forjudging  the  effects  of  changes  to 
the  training  program  provided  that  the  same  tests  are  used  to  measure  the  training  results. 
For  the  macro-level  manager,  the  test  data  would  be  much  more  valuable  if  they  could  be 
directly  equated  to  definable  levels  of  apprentice  or  journeyman  performance  in  the  field.  If 
equivalencies  could  be  established  then  it  would  become  possible  to  definitively  establish 
the  costs  of  training  to  specific  levels  of  competence.  The  impact  of  either  expanding  or 
reducing  formal  training  could  be  realistically  predicted,  thus  giving  the  training  manager  a 
very  powerful  tool  for  developing  and  justifying  policy  decisions. 

The  data  base  should  assist  the  training  manager  in  evaluating  the  impact  of  post¬ 
initial-skills  individual,  team,  and  unit  training  activities.  Studies  have  demonstrated  that 
flying  hours  are  directly  related  to  skilled  performances  such  as  carrier  landing  performance 
and  bombing  accuracy.  Other  studies  have  shown  that  time-to-repair  and  net  effectiveness 
ratings  are  useful  measures  for  evaluating  the  effects  of  post-initial-skills  training  of 
maintenance  personnel.  Much  of  this  kind  of  information  is  in  existing  data  banks.  Where 
feasible  these  data  should  be  considered  as  candidates  for  the  training  data  bank.  However, 
much  of  the  research  establishing  the  relationships  between  post-initial-skills  team  and  unit 
training  and  measures  of  performance  effectiveness  has  yet  to  be  done. 

D.  DATA  BASE  OPTIONS  AND  STRATEGIES 

The  options  for  building  a  data  base  to  support  the  Performance  Effectiveness 
aspect  of  a  training  system  are  constrained  by  the  limited  availability  of  the  necessary  kinds 
of  performance  data.  Probably  the  best  short-term  option  would  be  to  build  at  '  tg  data 
base  consisting  of  individual  personnel  selection  data,  training  histories,  and  operational 
achievement  data.  This  could  be  used  to  establish  a  historical  baseline  and  would  support 
at  least  limited  analyses  of  the  effects  of  the  introduction  of  changes  in  training  programs. 
It  should  also  be  possible  to  encourage  studies  that  systematically  extend  the  methodologies 
developed  for  the  cost-effectiveness  analysis  of  flight  simulators  to  other  simulation  and 
training  technology  evaluations.  This  would  serve  to  create  a  pool  of  cost-effectiveness 
data  for  estimating  the  training  benefits  to  be  expected  from  increased  utilization  of 
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technologically  advanced  training  options.  It  would  also  assist  in  establishing  guidelines 
for  the  routine  collection  of  essential  training  cost-effectiveness  data. 

On  an  intermediate  term  basis  it  should  be  possible  to  encourage  the  collection  of 
data  to  relate  training  course  parameters  (such  as  changes  in  course  length  or  the 
introduction  of  enhancements  to  training  courses)  to  students'  test  scores  and  changes  in 
user  satisfaction.  The  receiving  commands  could  be  tasked  to  provide  feedback  on  training 
effectiveness  with  these  students  by  routinely  producing  Net  Productivity  Ratings  of  first- 
term  personnel  assigned  to  the  command.  These  ratings  could  be  returned  to  the  training 
commands  and/or  included  in  the  individual's  permanent  records.  This  would  provide  a 
continuing  supply  of  data  for  cost-effectiveness  evaluations.  This  goal  could  be  achieved 
through  administrative  action  and  would  not  require  the  development  of  any  new 
technologies.  A  second  intermediate  option  would  be  to  identify  and  begin  to  collect  data 
relating  post-initial-skills  training  activities  (e.g.,  flying  hours  or  field  training  experience) 
to  the  maintenance  or  improvement  of  skills  while  performing  in  an  operational 
environment.  A  better  definition  of  the  functional  relationship  between  post-initial-skills 
training  and  both  the  short-term  and  cumulative  effects  on  skilled  performance  would  be 
very  useful  for  developing  and  justifying  training  policy. 

The  most  important  long-term  strategy  must  be  to  develop  performance  measures 
that  relate  training  achievement  levels  to  operational  apprentice  or  journeyman  skill  levels. 
If  this  can  be  achieved,  it  will  be  possible  to  develop  meaningful  cost-effectiveness  trade¬ 
offs  for  training.  The  previously  discussed  Net  Productivity  Ratings  partially  fulfill  this 
objective.  However,  they  have  two  limitations:  (1)  they  require  a  minimum  delay  of 
18  months  to  two  years  in  assessing  the  effects  of  training  programs  on  the  quality  of 
performance,  and  (2)  student  and  supervisor  productivity  information  are  confounded, 
which  partially  masks  the  separate  effects  of  training  on  student  performance. 

A  scale  relating  training  achievement  to  operational  skill  levels  would  provide 
immediate  feedback.  It  could  also  be  used  to  determine  the  costs  of  training  to  a  specified 
skill  level.  This  would  make  many  training  decisions  more  directly  analogous  to  hardware 
cost-effectiveness  trade-offs  in  which  the  costs  of  training  to  a  specified  criterion  could  be 
weighed  against  the  anticipated  operational  benefits.  The  long-term  strategy  should  also 
encourage  the  development  and  validation  of  highly  aggregated  measures  of  the  effects  of 
collective  training  (e.g.,  collective  training  as  a  force  multiplier). 
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VI.  READINESS  EVALUATION  FUNCTIONS 
IN  TRAINING 


The  effects  of  training  on  combat  readiness  (when  demonstrable)  would  provide  the 
most  compelling  basis  for  formulating  training  policy  decisions.  This  aspect  of  the 
Training  Management  Model  represents  the  effects  of  training  on  individual  and  group 
combat  readiness.  Readiness  evaluation  essentially  verifies  that  the  required  numbers  and 
types  of  personnel  have  been  trained  and  that  they  are  capable  of  performing  their  unit's 
specified  mission.  Readiness  evaluations  may  be  classified  into  two  categories: 
prescriptive  and  empirical.  The  prescriptive  readiness  evaluations  serve  an  essential 
administrative  and  accounting  function  because  they  seek  to  verify  that  the  approved 
resources  have  produced  the  required  training  products.  The  empirical  readiness 
evaluations  seek  to  determine  the  effects  of  training  upon  how  well  an  operational  unit  can 
perform  its  assigned  mission.  The  current  readiness  reporting  systems  within  DoD  are 
predominantly  verifications  of  prescriptive  compliance.  A  unit  is  considered  combat  ready 
if  the  prescribed  complement  of  personnel  and  materiel  are  in  place  and  the  formally 
prescribed  individual  and  collective  training  courses  and  events  have  been  completed. 
A  unit's  readiness  status  is  a  joint  function  of  the  resource  and  training  status  at  the  time  of 
the  evaluation  tempered  by  an  estimate  of  the  amount  of  time  needed  to  complete  any 
unfulfilled  resource  and  training  requirements. 

Empirical  readiness  measures  which  relate  training  to  the  quality  of  combat 
performance  should  have  the  greatest  face  validity  and  the  greatest  impact  on  training 
policy.  A  dramatic  example  of  the  effects  of  combat  performance  data  on  training  policy  is 
presented  in  the  development  of  realistic  air  combat  training  programs  in  the  Services.  The 
Navy  Fighter  Weapons  School  (Top  Gun)  was  established  because  of  the  relatively  poor 
performance  of  American  pilots  in  air  combat  during  the  Vietnam  War. 

Top  Gun  was  extremely  effective  in  training  Navy  fighter  jocks  for  air-to-air 
combat.  For  the  Navy,  the  air-to-air  scorecard  over  Southeast  Asia  could  be 
split  into  two  segments:  before  Top  Gun-2: 1,  and  after  Top  Gun- 13:1. 
(Skinner,  1984) 


With  the  exception  of  some  limited  activity  in  the  Persian  Gulf,  U.S.  forces  are  not 
now  engaged  in  active  combat;  consequently,  it  is  no  longer  possible  to  evaluate  the  effects 
of  this  type  of  advanced  realistic  training  on  operational  performance.  Surrogate  measures 
from  war  games,  large  scale  simulations,  and  operational  performance  data  are  used  to 
estimate  the  effect  that  training  is  likely  to  have  on  combat  capabilities.  Empirical  readiness 
evaluations  in  the  TTM  serve  the  purpose  of  justifying  the  establishment  or  modification  of 
training  programs.  Once  established  or  appropriately  modified,  a  training  program  is 
assumed  to  be  effective  unless  proven  otherwise.  If  effectiveness  is  assumed,  then  the 
major  management  concerns  are  throughput  and  cost.  Questions  of  how  many  people  are 
trained  and  how  much  it  costs  were  presented  in  Chapter  IV  (Cost-Production).  The 
Readiness  Evaluation  function  of  the  TMM  focuses  on  the  effects  of  training  on  measures 
considered  either  representative  or  predictive  of  combat  effectiveness. 

The  Readiness  Evaluation  function,  like  the  Performance  Effectiveness  function, 
has  some  important  limitations.  Pate  (1975)  reviewed  the  historical  development  of  the 
readiness  reporting  system  with  special  emphasis  on  the  impact  of  the  readiness  reports  on 
the  Army  Reserve  Components  (RCs).  Two  of  his  comments  are  of  particular  significance 
to  our  present  discussion.  First,  readiness  evaluations  in  current  use  are  subjective 
judgments  of  a  unit’s  combat  readiness  as  viewed  by  its  commanding  officer. 

In  its  totality,  force  readiness  is  a  subjective  "judgment  call"  and  not 
amenable  to  any  form  of  statistical  quantification." 

Pellicci  (1985)  also  refers  to  the  subjective  nature  of  reporting  readiness. 

The  current  measure  of  training  status  is  subjectively  derived.  Commanders 
have  little  objective  criteria  on  which  to  assess  their  overall  unit  training 
readiness,  and  inadequate  methodology  to  ensure  consistency.. .two 
identical  units  trained  to  the  same  proficiency  standard  could  easily  report 
totally  different  ratings. 

Second,  readiness  evaluations  do  not  apply  to  all  of  the  units  and  organizations 
within  the  Services.  With  respect  to  the  Army,  Pate  noted: 

Roughly  half  of  the  Army's  total  manpower  (the  most  costly  recurring  item 
in  the  Army  budget)  is  committed  to  organizations  and  activities  which  do 
not  even  render  unit  readiness  reports. 

(e.g.,  headquarters,  supply,  training,  and  R&D  organizations). 

The  importance  of  readiness  measures  should  not  be  underestimated  even  though 
they  have  some  significant  limitations.  Collectively,  they  provide  the  single  best  estimate 
of  the  proportion  of  the  armed  forces  that  are  considered  to  be  fully  capable  of  performing 
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their  organizational  role  or  function  at  any  given  point  in  time.  Readiness  evaluations  are 
used  by  Congress  to  evaluate  future  needs  and  to  assess  past  accomplishments.  (A  typical 
question  from  Congress  takes  the  form  of  "How  much  money  do  you  need  next  year  and 
what  did  you  do  with  the  money  that  we  gave  you  last  year?")  Data  that  could  clearly 
establish  a  functional  relationship  between  training  resources  and  combat  readiness  would 
be  of  considerable  value  to  the  macro-level  training  manager. 

The  Readiness  Evaluation  function  of  training  system  operation  is  presented  in 
Figure  6.  The  readiness  evaluation  function  is  represented  by  the  box  labeled  "Readiness". 
In  keeping  with  the  character  of  readiness  evaluations,  there  is  more  than  one  way  to 
achieve  "Readiness".  The  first  route  which  is  used  for  prescriptive  readiness  evaluations 
provides  interconnections  between  "Requirements",  "Resources",  and  "Readiness".  The 
requirements  specify  the  numbers  and  types  of  personnel  and  training  programs;  resources 
fund  the  needed  personnel,  facilities  and  equipment;  readiness  evaluation  essentially 
verifies  that  the  required  numbers  and  types  of  personnel  have  been  trained  and  are  ready  to 
perform  their  unit's  specified  mission.  The  requirements  represent  the  assigned  obligation 
and  the  readiness  measures  represent  the  current  ability  to  accomplish  that  obligation.  Any 
differences  between  requirements  and  readiness  status  will  serve  as  an  input  to  modify 
requirements. 

The  second  route  provides  the  linkage  between  training  and  empirical  readiness 
evaluations.  Part  of  the  readiness  evaluation  is  based  upon  the  quality  of  personnel 
performance  or  at  least  upon  the  effects  of  the  knowledge  and  skills  gained  through 
training.  Any  persistent  and  inherent  shortcomings  in  performance  are  likely  to  be  reflected 
back  into  a  revision  of  the  requirements  (or  perhaps  in  the  development  of  better 
equipment).  Since  only  part  of  the  training  will  directly  influence  combat  readiness  (only 
part  of  the  training  effort  is  directed  toward  the  development  of  combat  and  combat  support 
skills),  only  part  of  the  training  output  goes  to  the  "Readiness"  box  while  the  rest  provides 
direct  input  into  the  "Requirements"  box. 

While  providing  a  source  of  potentially  valuable  information  to  the  macro-level 
training  manager,  Readiness  Evaluation  functions  primarily  to  supplement  the  Cost- 
Production  function.  In  some  ways  it  serves  as  a  quality  control  system  similar  to  the 
quality  control  functions  discussed  in  Performance  Effectiveness.  If  the  Performance 
Effectiveness  function  can  be  thought  of  as  representing  the  training  producer's  quality 
control  efforts,  the  Readiness  Evaluation  function  can  be  thought  of  as  the  consumer's 
quality  control  assessment. 
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Figure  6.  Readiness  Evaluation  Functions  of  Training 
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Data  relevant  to  Readiness  Evaluation  will  be  presented  in  four  categories: 

1 .  Measures  of  readiness 

2.  Training  inputs 

3.  Performance  assessment 

4  Effects  of  training  on  readiness. 

A.  MEASURES  OF  READINESS 

Ideally,  measures  of  training  effectiveness  should  document  the  functional 
relationship  between  training  and  the  ability  of  a  unit  to  accomplish  its  combat  role  or 
mission.  Measures  of  effectiveness  of  training  must  ultimately  be  related  to  measures  of 
military  effectiveness  and  worth.  A  first  step  toward  this  kind  of  a  goal  can  be  achieved  by 
relating  training  to  measures  of  unit  readiness. 

DOD  Directive  5100.30:  Worldwide  Military  Command  and  Control  System 
(WWMCCS)  (1971)  directs  the  National  Military  Command  Center  to  maintain  the  current 
status  of  U.S.  operational  forces,  including  readiness  posture.  Cavalluzzo  (1985),  in 
discussing  the  Navy's  procedures  for  reporting  readiness  status  stated  that 

"The  Unit  Status  and  Identity  Reporting  System  (UNITREP)  is  the  official 
reporting  mechanism  used  to  convey  timely  readiness  information  to  the 
Joint  Chiefs  of  Staff.  Readiness  is  reported  in  four  dimensions:  personnel, 
training,  materiel  condition  and  supplies  on  board.  The  UNITREP  Combat 
Readiness  Training  Index  (CRTRNG)  is  a  composite  measure,  relative  to 
prescribed  requirements,  of  shipboard  exercises  satisfactorily  completed, 
personnel  who  have  attended  formal  schools,  and  personnel  qualified  to 
oversee  watch-stations." 

Pellicci  (1985)  noted  that  battalion  commanders  report  their  training  readiness  status 
to  Headquarters,  Department  of  the  Army  (HQDA)  on  a  monthly  basis  via  the  Unit  Status 
Report  (USR).  The  estimate  of  training  readiness  is  reported  in  terms  of  a  combat 
readiness  code  ranging  from  C-l  (Combat  ready)  through  C-4  (Not  combat  ready)  (see 
Table  4). 

Pate  (1975),  in  regards  to  the  Reserve  Component  capability  assessment  system, 
indicated  that  the  time  needed  to  get  a  unit  "ready"  should  be  compared  to  the  time  allowed 
for  the  same  unit  to  get  "ready"  to  determine  whether  the  unit  is  at  a  satisfactory  level  of 
readiness.  (Pate's  time  to  get  a  unit  "ready"  includes  the  estimated  time  required  to  fill 
equipment  and  personnel  shortages  and  to  complete  the  prescribed  unit  training  exercises.) 


Table  4.  Combat  Readiness  Ratings8*1* 


C-1 

Combat  ready 

C-2 

Combat  ready,  minor  deficiencies 

C-3 

Combat  ready,  major  deficiencies 

C-4 

Not  combat  ready 

8  Cavalluzzo,  1985. 
b  Pellicci,  1985 


This  is  essentially  a  "ready  enough"  concept;  i.e.,  a  unit  that  is  not  scheduled  for 
mobilization  until  60  days  after  the  start  of  hostilities  should  be  considered  ready  enough  if 
it  can  be  brought  up  to  C-l  status  within  60  days. 

The  Standards  in  Training  Commission  (STRAC)  was  created  to  establish  Army- 
wide  gunnery  training  strategies.  As  part  of  this  effort,  units  were  classified  on  the  basis  of 
their  deployment  requirements  as  specified  in  the  Army  Mobilization  and  Operations 
Planning  System  (AMOPS).  The  STRAC  Training  Readiness  Condition  (TRC) 
classification  system  assigned  units  to  one  of  four  levels  (A-D)  based  upon  the  number  of 
days  available  until  deployment.  Combining  the  STRAC  TRC  ratings  with  the  USR 
combat  readiness  structure  produces  a  system  which  indicates  the  minimum  combat 
readiness  status  that  a  unit  should  maintain  based  upon  its  deployment  schedule. 
The  interrelationship  of  the  two  classification  systems  is  presented  in  Figure  7. 
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TRC  A 

TRC  B 

TRC  C 

TRCD 
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USR  C-Ratina 

C-1 
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Weeks  to  Train 

1-2 

3-4 
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6+ 

Source:  Pellicci,  1985,  p.  341 


Figure  7.  Training  Readiness  Condition  and  Status  Classification  Systems 
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(It  should  be  noted  that  the  time  intervals  used  in  the  two  systems  differ  in  terms  of  their 
basic  units  of  measure,  days  versus  weeks,  and  that  the  specified  time  intervals  diverge 
slightly  over  time.)  Systematically  relating  training  needs  to  deployment  plans  would  seem 
to  provide  a  basis  for  specifying  the  minimum  kinds  and  amounts  of  training. 
Conceptually,  at  least,  this  kind  of  approach  should  also  be  useful  for  evaluating  the  effects 
of  either  increases  or  decreases  in  training  resources. 

B.  TRAINING  INPUTS 

Any  systematic  effort  to  relate  training  activities  to  readiness  criteria  needs  some 
measure  of  training  input.  Most  training  inputs  will  be  measured  in  terms  of  dollars, 
personnel,  or  time.  While  most  of  the  training  input  measures  have  been  presented  in 
considerable  detail  in  our  discussion  of  the  Production-Cost  function,  some  aspects  of  the 
input  measures  merit  further  consideration  in  our  discussion  of  the  Readiness  Evaluation 
function.  The  discussion  of  training  inputs  will  be  divided  into  the  areas  of  individual  and 
collective  training.  Attention  will  be  focused  on  the  value  of  relating  training  inputs  to 
readiness  goals. 

1.  Individual  Training  Inputs 

The  DOD  Force  Readiness  Report  for  FY  1984  (DOD,  1983)  stated  that  the  training 
readiness  of  air  crews  is  affected  by  the  proportion  of  billets  filled  by  trained  individuals 
(which  is  a  function  of  the  individual  training  process),  and  the  level  of  unit  proficiency 
obtained  (usually  measured  as  a  percent  of  available  combat-qualified  air  crews).  Although 
this  statement  is  concerned  with  the  effects  of  individual  training  upon  air  crews,  it  seems 
reasonable  to  believe  that  individual  training  contributes  equally  well  to  the  readiness  status 
of  other  military  groups  or  units. 

The  management  of  training  is  concerned  with  producing  the  appropriate  numbers 
and  types  of  personnel  needed  to  satisfy  readiness  requirements.  One  of  the  first 
considerations  for  the  macro  level  manager  is  to  know  how  many  and  what  kinds  of  trained 
personnel  are  being  produced.  The  appropriate  measure  for  this  question  is  the  student 
training  load  as  reported  for  various  categories.  Student  training  loads  are  usually 
represented  in  terms  of  the  total  person-years  of  individual  training  contained  in  the  Service 
budgets. 

Once  the  amount  of  training  in  person-years  is  known,  the  next  question  usually  is 
whether  that  is  enough.  In  response  to  this  question  the  measure  of  choice  would  appear  to 
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be  projections  of  personnel  inventory,  derived  from  projections  for  personnel  needed  for 
weapon  systems,  personnel  rotation,  and  so  forth.  These  projections  add  another 
dimension  to  the  student  load  data  by  indicating  whether  the  Services  plan  to  support  the 
training  of  enough  people  in  needed  skills. 

Training  readiness  reports  provide  a  snapshot  of  the  training  readiness  of  combat 
units.  By  tracking  student  training  loads,  personnel  inventory  projections,  and  readiness 
reports  over  time,  it  should  be  possible  to  identify  functional  relationships  among  the 
measures  which  could  be  used  as  training  management  tools.  Knowledge  of  these 
functional  relationships  coupled  with  data  from  appropriately  structured  data  banks  would 
let  the  macro  level  manager  formulate  and  answer  a  number  of  "What  if'  questions.  For 
example,  if  training  loads  go  down  due  to  training  cuts  during  one  FY  what  happens  to 
training  readiness  in  the  succeeding  FYs? 

2 .  Collective  Training  Inputs 

As  has  been  previously  discussed,  the  collective  training  inputs  can  be  monitored  in 
terms  of  flying  hours,  steaming  days,  and  battalion  training  days.  Of  interest  here  is  the 
potential  benefit  of  tying  the  input  measures  to  readiness  goals. 

Based  on  the  literature  reviewed,  the  relationship  between  collective  training  and 
readiness  goals  appears  to  be  rather  vague  and  nebulous.  Cavalluzzo  (1985)  commented 
that  prior  to  her  exploratory  work,  no  one  from  the  analytical  community  had  tried  to 
measure  the  payoff  of  the  collective  training  expenditures  (steaming  days)  in  terms  of 
readiness.  Her  work  does  indicate  that  there  was  a  positive  relationship  between 
nondeployed  activity  and  training  readiness. 

The  most  extensive  effort  to  relate  collective  training  input  to  readiness  appears  to 
be  the  Army  Flying  Hour  Program  (FHP).  As  presented  in  the  DOD  Force  Readiness 
Report  for  FY  1984  (DOD,  !Q33), 

The  Active  Army  Flying  Hour  Program  provides  the  minimum  number  of 
flying  hours  to  support  individual  training  and  to  maintain  unit  level 
proficiency  for  TOE  aviation  units  at  their  programmed  manning  levels. 

Pellicci  (1985)  provided  further  insight  into  the  structure  and  benefits  of  the  FHP. 

Gradually  the  program  matured  and  expanded  beyond  meeting  the  published 
annual  training  requisites  of  the  individual  pilot  and  to  embrace  the  full 
spectrum  of  requirements;  at  the  individual,  crew,  unit  and  combined  arms 
technical  and  tactical  proficiency  levels. 
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The  benefits  of  relating  training  inputs  to  specific  readiness  requirements  can  be 
seen  in  some  of  the  results  of  the  FHP.  Pellicci  noted  that 

The  FHP  identified  a  serious  flying  hour  and  training  readiness  shortfall 
projected  to  worsen  in  the  near  term.  The  trend  was  reversed  during  the 
ensuing  programming  and  budgeting  cycle,  and  accelerated  by  a  further 
Congressional  add-on  when  the  Army's  model-developed  requirements-for 
each  type  of  aircraft- were  laid  out 

In  those  situations  in  which  a  strong  and  highly  specific  relationship  can  be 
established  between  a  training  input  and  a  readiness  goal,  the  likelihood  of  getting  the 
necessary  resources  seems  to  be  enhanced.  While  the  availability  of  the  right  kinds  of  data 
may  be  limited,  it  would  appear  desirable  to  promote  the  development  of  training  models 
and  training  data  banks  that  would  more  systematically  relate  collective  training  inputs  to 
training  readiness  objectives. 

3 .  Performance  Assessment 

While  the  quality  of  individual  performance  unquestionably  influences  unit 
performance,  the  focus  in  any  readiness  evaluation  is  on  the  overall  ability  of  the  unit  to 
accomplish  its  mission.  Consequently,  the  primary  concern  in  readiness  evaluations  is  unit 
performance. 

Although  the  Services  have  been  been  refining  their  readiness  evaluation  systems 
for  years,  analytical  attempts  to  relate  training  variables  to  readiness  measures  seem  to  have 
evolved  only  recently.  This  section  will  present  some  candidate  performance  measures 
related  to  readiness.  Some  of  the  results  of  trying  to  relate  training  data  to  readiness-related 
performance  measures  are  presented  next 

Horowitz  (1986)  identified  and  discussed  a  number  of  performance  measures  for 
possible  use  in  formulating  and  supporting  Navy  manpower,  personnel,  and  training 
(MPT)  policies.  Table  5  summarizes  his  list  of  Navy  performance  measures.  Although  his 
primary  concern  was  with  the  Navy,  he  also  suggested  several  performance  measures  for 
the  Army.  These  included:  data  from  the  Army  Training  Evaluation  Program  (ARTEP), 
information  from  the  National  Training  Center  (NTC)  combat  exercises,  and  scores  from 
the  Skill  Qualification  Tests  (SQTs). 

While  this  list  of  performance  measures  is  not  exhaustive,  it  suggests  that  there  are 
a  number  of  ways  in  which  training  data  might  be  related  to  some  meaningful  estimate  of 
readiness. 


55 


Table  5.  Navy  Inspections  and  Examinations  as  Potential  Sources  of 
Performance  Data  for  Evaluating  the  Effects  of 
Training  on  Readiness8 


Operational 

Performance 


Maintenance 

Performance 


individual 

Performance 


Training  Inspections 
Source  of  information  on  the 
performance  of  units  in  combat¬ 
like  situations. 

Operational  Propulsion 
Plant  Examinations  (OPPEs)-- 
an  objective  measure  of  ship 
performance  in  the  area 
of  mobility. 


Casualty-Bfiports 

(QASREPs) 

Whenever  a  ship  suffers  an 
equipment  failure  that 
adversely  affects  its  ability 
to  cany  out  its  primary 
missions  (and  it  cannot  be 
repaired  within  48  hours), 
it  is  supposed  to  file  a 
casualty  report. 


Scores  on  Advancement 
Examinations 
To  be  promoted  to  any 
pay  grade  above  E-3  an 
individual  must  pass  an 
examination  in  the 
skills  that  must  be 
mastered  to  perform 
successfully  in  that 
higher  pay  grade. 


Ship  ASW  Readiness 
Effectiveness  Exercise 
(SHAREMs)— measure  of  ship 
proficiency  in  antisubmarine 
warfare. 

Operational  Readiness 
Evaluations  (OREs)-tests  of 
operational  performance. 

Selected  Exercises 
Evolutions  performed  by 
ships  as  part  of  their 
regular  training  syllabi. 


Excellence  Awards 
Scores  on  Selected 
Exercises  are  aggregated 
to  help  determine  the  winners 
of  the  mission-area- 
excellence  awards  held  by 
squadron  commanders. 


Unit  Status  and  Identify 
Reporting  System  (UNITREPI 
Periodic  status  reports  of 
units  in  each  of  four  resource 
categories: 
personnel,  training, 
supply  and  material. 


Form  4855 

Information  on  equipment 
logs  for  selected  combat 
systems  -  when  equipment 
was  on,  when  failure  was 
detected,  and  when  failure 
was  fixed. 

Board  Qllnspecticn 

and  Surrey  (1NSUBVI 
INSURV  Board  inspects 
1 00  ships  a  year-the  principal 
indicator  of  material  readiness 
is  the  Materials  Condition 
Index  (MCI). 


Promotion  Decisions 
Whether  or  not  someone 
is  promoted  is  probably 
the  best  single  summary 
measure  of  how  highly  he 
is  regarded  in  relation  to 
other  individuals  who  are 
eligible  for  consideration. 


a  Horowitz,  1986. 


(Continued) 
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Table  5  (cont'd).  Navy  Inspections  and  Examinations  as  Potential  Sources  of 
Performance  Data  for  Evaluating  the  Effects  of 
Training  on  Readiness8 


Operational 

Performance 

Maintenance 

Performance 

Individual 

Performance 

Bombing, .ScQf.es 

Attack  squadrons  keep  track 
of  the  frequency  and  accuracy 
of  different  types  of  practice 
bomb  runs. 

Tactical  Aircrew 

Combat  System  (TACTS) 

Air  combat  maneuvering 
ranges  provide  information  on 
flight  dynamics,  weapon 
system  states  and  weapon 
firing  of  each  aircraft  engaged 
in  a  training  mission.  (Data  is 
held  extremely  closely.) 

Simulators 

Information  on  the  operational 
performance  of  units  and 
personnel  can  be 
supplemented  by  data  on  their 
proficiency  using  advanced 
training  devices. 

P-3  squadrons- 
individuals  and  crews  as  a 
whole  are  graded  on 
performance. 

ASW,  AAW,  and  EW 
simulators  could  provide 
performance  measures  of  ship 
teams. 

a  Horowitz,  1986. 
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4.  Effects  of  Training  on  Readiness 

In  discussing  the  relationship  of  training  to  unit  readiness,  it  should  be  remembered 
that  training  is  only  one  component  in  the  readiness  equation.  The  effects  of  training  may 
be  inextricably  confounded  with  the  effects  of  personnel  and  equipment  The  DOD  Force 
Readiness  Report  for  FY  1984  (DOD,  1983)  noted  that  the  level  of  programmed  manning 
also  has  a  direct  impact  on  training  readiness,  because  the  ability  of  a  unit  to  conduct 
meaningful  collective  training  decreases  rapidly  as  strength  drops.  This  is  particularly  true 
for  units  that  rely  heavily  on  team  skills  (bomber  and  tank  crews,  infantry  squads,  ship 
damage  control  parties),  where  the  absence  of  even  a  few  members  can  severely  degrade 
effective  training. 

However,  in  spite  of  limitations  in  the  quality  of  data  and  the  number  of  studies  that 
have  tried  to  relate  training  to  readiness,  there  is  just  enough  data  to  suggest  that  this  would 
be  a  fruitful  area  for  further  investigation.  As  was  discussed  earlier,  Cavalluzzo  (1985) 
found  a  consistently  positive  relationship  between  flight  hours  and  ORE  scores.  She  also 
found  that  an  increase  in  the  number  of  days  per  quarter  spent  in  training  was  associated 
with  an  increase  in  the  number  of  ships  that  were  combat  ready  when  they  began  their 
deployment 

Using  Casualty  Report  (CASREP)  down  time  as  their  measure  of  personnel 
productivity  and  equipment  readiness,  Horowitz  and  Angier  (1985)  reported  that  the 
quantity  of  formal  Navy  training  completed  by  the  ship's  maintenance  personnel  was  a 
consistent  predictor  of  ship  readiness.  Rowland  (1985)  reported  that  two  weeks  of 
experience  in  simulated  interactive  combat  increased  troop  proficiency  from  an  initial 
50  percent  level  to  the  95  percent  level  (measured  in  exchange  ratios  between  casualties 
inflicted  and  taken). 

While  readiness  data  is  not  a  component  of  training  data  banks,  it  appears  likely  that 
there  may  be  a  significant  amount  of  readiness  data  kept  in  other  types  of  Service  data 
banks  which  can  be  used  to  verify  the  effects  of  training  on  readiness. 

C.  DISCUSSION  AND  CONCLUSIONS 

In  order  for  the  data  base  to  support  policymaking  efforts  under  the  Readiness 
Evaluation  function  of  training,  it  should  be  able  to  provide  information  concerning  the 
degree  that  the  prescriptive  readiness  requirements  have  been  met  or  else  provide 
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information  concerning  the  effects  of  training  on  combat  performance  or  other  appropriate 
operational  measures. 

The  Readiness  Evaluation  function  of  the  TMM  serves  two  functions:  (1)  it 
provides  a  means  for  determining  how  well  the  prescriptive  readiness  requirements  have 
been  met,  and  (2)  it  has  the  potential  for  providing  the  ultimate  test  of  training 
effectiveness— an  assessment  of  the  effects  of  training  or  training  changes  on  combat 
performance.  However,  like  the  Performance  Effectiveness  function,  its  prime  purpose  is 
to  mandate  or  evaluate  changes  in  the  training  program.  Once  the  changes  have  been  made 
and  evaluated,  the  continuing  assessment  of  training  is  primarily  a  Cost-Production  issue. 

The  data  base  should  assist  the  training  manager  in  monitoring  the  training  system’s 
level  of  success  in  providing  the  required  amounts  of  training.  Measures  of  personnel 
readiness  provide  an  indication  of  whether  the  system  is  producing  adequate  numbers  of 
trained  personnel.  Measures  of  training  readiness  can  indicate  whether  the  training  time 
and  resources  have  been  sufficient  to  complete  the  required  number  of  collective  training 
events.  Ideally  the  training  manager  should  have  access  to  data  which  demonstrate  the 
effects  of  training  on  combat  performance.  However,  there  is  currently  very  little  data 
available  to  establish  the  relationships  between  training  and  combat  performance.  Efforts  in 
the  areas  of  war  games,  large-scale  simulations,  networked  simulations,  and  efforts  to 
relate  training  to  currently  used  achievement  measures  promise  to  improve  significantly  the 
ability  of  the  training  community  to  relate  training  experience  empirically  to  performance  in 
near-combat  situations. 

D .  DATA  BASE  OPTIONS  AND  STRATEGIES 

Building  the  data  base  needed  for  Readiness  Evaluation  of  the  training  system  faces 
some  serious  obstacles.  The  best  short-term  option  would  appear  to  be  one  which 
concentrates  heavily  on  obtaining  prescriptive  readiness  data.  The  data  exist,  they  are 
collected,  and  they  are  used  by  the  Services  and  DoD  in  reporting  to  Congress  on  the  state 
of  military  readiness.  Relating  training  budgets  to  the  training-related  aspects  of  readiness 
over  time  should  provide  a  measure  of  the  training  system's  effectiveness  and  the 
relationship  between  training  budgets  and  readiness. 

Developing  a  data  base  to  evaluate  the  effects  of  training  on  the  level  of  empirical 
readiness  is  severely  limited  by  the  availability  of  data.  At  the  present  time  this  is  more  of  a 
research  problem  than  a  data  base  problem.  In  the  short  term  an  effort  should  be  made  to 
collect  and  summarize  existing  information  on  the  relationship  between  training  and 


performance  in  combat  or  combat-like  situations  (possibly  going  back  to  WW I  or  earlier). 
This  would  at  least  allow  the  training  manager  to  show  situations  in  which  training  policy 
decisions  have  made  a  significant  difference;  e.g.,  the  change  in  kill  ratios  before  and  after 
the  establishment  of  the  Navy  Fighter  Weapons  School.  In  the  intermediate  time  frame 
analytical  efforts  such  as  those  suggested  by  Horowitz  (1986)  should  be  tracked  and 
evaluated  for  use  by  training  managers.  On  a  longer  term,  efforts  to  assess  more 
adequately  the  effects  of  training  in  simulated  combat,  networked  simulations,  and  war 
games  should  be  encouraged  and  supported. 
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VII.  SUMMARY  AND  CONCLUSIONS 


The  primary  purpose  of  this  paper  has  been  to  provide  guidance  for  identifying  the 
types  of  data  that  the  Macro-Level  Cost  Data  Bases  should  contain  to  be  useful  for  making 
decisions  in  training  policy.  If  the  desired  data  base  information  can  be  specified,  it  will 
then  be  possible  to  assess  whether  the  current  data  bases  contain  the  needed  information 
and  to  recommend  necessary  improvements.  The  Training  Management  Model  (TMM) 
provides  a  global  description  of  the  functioning  of  the  training  system  from  the  perspective 
of  the  macro-level  training  manager.  Based  upon  the  TMM  and  a  survey  of  the  available 
training  performance  data,  the  types  of  data  required  for  the  Macro-Level  Cost  Data  Bases 
have  been  presented.  When  the  various  data  base  options  and  strategies  are  considered,  the 
results  will  provide  a  master  plan  for  macro-level  training  data  base  development. 

The  TMM  describes  the  training  system  and  emphasizes  three  of  its  main  functions. 
Each  of  these,  i.e.,  Cost-Production,  Performance  Effectiveness,  and  Readiness 
Evaluation,  consists  of  sets  of  resource,  training,  and  evaluation  elements  in  a  continuous 
feedback  loop.  These  provide  the  manager  with  the  data  needed  to  be  able  to  respond  to 
specific  types  of  problems  that  arise  from  time  to  time. 

A.  COST-PRODUCTION  FUNCTION 

Policymaking  efforts  related  to  Cost-Production  require  information  regarding  the 
allocation  of  training  funds,  costs  of  training,  levels  of  effort,  and  measures  of  training 
system  proficiency.  Data  describing  the  funds  allocated  to  training  and  training  costs  are 
available;  however,  whether  they  are  the  right  kind  or  are  in  the  appropriate  formats  has  yet 
to  be  determined.  Measures  of  levels  of  effort  in  the  form  of  student  training  loads  and 
inventory  projections  for  individual  training  and  flying  hours,  steaming  hours,  and 
battalion  training  days  for  collective  training  can  be  obtained.  Some  measures  of  system 
proficiency  such  as  training  program  completions,  productivity  estimates  and  training 
readiness  reports  are  either  available  or  could  be  obtained. 

The  Cost- Production  function  of  the  TMM  represents  the  normal  management  of 
the  training  system.  Based  upon  approved  sets  of  requirements,  resources  in  the  form  of 


budgets,  facilities,  and  personnel  are  channeled  into  individual  and  group  training  programs 
which  ultimately  serve  to  fulfill  or  modify  the  existing  requirements.  Two  assumptions  are 
implicit  in  the  Cost-Production  function:  (1)  The  training  courses  and  events  are  effective 
in  conveying  the  necessary  information  and  skills  to  the  personnel  involved,  and 
(2)  Trained  personnel,  teams,  and  units  advance  the  state  of  readiness  and  potential  combat 
effectiveness  of  the  armed  forces.  When  these  assumptions  have  questionable  validity,  the 
training  manager  needs  to  obtain  appropriate  performance  or  readiness  data. 

The  macro-level  training  manager  is  also  concerned  with  the  management  of  the 
capabilities  and  capacities  of  the  training  system.  This  is  the  macro-level  training 
manager's  unique  sphere  of  influence.  Descriptive,  trend,  and  functional  relationship  data 
are  needed  to  manage  present  training  capabilities  and  capacities  and  to  plan  for  the  future 
characteristics  of  the  training  system.  There  is  a  consistent  need  to  be  able  to  summarize 
capabilities,  needs  and  trends  in  order  to  formulate  and  defend  training  policy. 

B.  PERFORMANCE  EFFECTIVENESS  FUNCTION 

The  Performance  Effectiveness  function  of  the  TMM  supplements  the  more  routine 
Cost-Production  function  in  several  ways:  (1)  Within  training  courses  individual  perfor¬ 
mance  data  support  the  training  quality  control  process  and  provide  the  basis  for  deciding 
whether  an  individual  should  be  continued  into  the  next  phase  of  training,  graduated, 
recycled,  or  attrited.  (2)  Data  on  group  performance  within  systems  (training  or 
operational)  provide  a  basis  for  evaluating  the  effectiveness  of  training  courses  or 
techniques.  A  positive  relationship  between  training  and  readiness  and  potential  combat 
effectiveness  is  implicitly  assumed. 

The  Performance  Effectiveness  function  provides  the  macro-level  training  manager 
with  information  on  the  level  of  skilled  performance  produced  by  the  training-initial  skills 
training,  on-the-job  training,  post-initial  skills  individual  training,  and  team  and  unit 
training.  The  performance  effectiveness  data  needed  for  policy  development  can  be  divided 
into  two  categories:  (1)  data  which  relate  student  achievement  levels  realized  in  formal 
courses  to  the  skill  levels  required  by  the  operating  forces,  and  (2)  data  which  relate  post¬ 
initial  skills  training  activities  such  as  flying  hours  to  maintaining  or  improving  the  skill 
levels  of  personnel  while  they  are  performing  in  the  operational  environment. 
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C.  READINESS  EVALUATION  FUNCTION 


The  Readiness  Evaluation  function  of  the  TMM  supplements  the  Cost-Production 
function  by  verifying  that  the  required  numbers  and  types  of  personnel  have  been  trained 
and  that  they  are  capable  of  performing  their  their  unit's  specified  mission.  The  macro¬ 
level  training  manager  requires  both  prescriptive  and  empirical  readiness  data.  The 
prescriptive  data  verify  that  the  approved  resources  have  been  used  to  produce  the  required 
training  products.  The  empirical  readiness  data  verify  the  effects  of  training  on  the  ability 
of  operational  units  to  perform  their  assigned  missions.  Since  readiness  evaluation  data 
present  acceptable  proof  of  either  the  ultimate  effectiveness  or  the  inadequacies  of  training, 
they  provide  the  most  compelling  basis  for  formulating  and  supporting  training  policy 
decisions.  However,  like  the  Performance  Effectiveness  function,  the  prime  purpose  of  the 
Readiness  Evaluation  function  is  to  initiate  or  manage  the  process  of  change  in  the  training 
program.  Once  the  changes  have  been  approved,  implemented,  and  verified,  the 
assessment  of  training  is  primarily  a  Cost- Production  issue. 

D .  TRAINING  MANAGEMENT  MODEL  (TMM) 

The  TMM  provides  a  description  of  the  operation  of  the  military  training  system.  It 
gives  macro-level  training  managers  a  tool  to  help  them  specify  the  data  and  data  bases 
needed  to  develop  and  support  training  policy  decisions.  Based  on  the  TMM  and  a  survey 
of  the  available  training  performance  data,  a  series  of  data  base  options  and  strategies  were 
presented.  These  provide  the  basis  for  the  construction  of  a  master  plan  for  training  data 
base  development  which  is  presented  in  the  Recommendations  section. 
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VIII.  RECOMMENDATIONS 


The  Training  Management  Model  (TMM)  and  its  three  major  functions  provide  the 
basis  for  identifying  the  types  of  data  that  a  macro-level  training  manager  will  need  to 
formulate  and  support  training  policies  in  the  areas  of  the  cost,  effectiveness,  and  cost- 
effectiveness  of  producing  trained  personnel;  the  management  of  training  system 
capabilities,  capacities,  and  utilization  rates;  the  effects  of  training  on  the  quality  of 
individual  and  unit  performance;  and  the  effects  of  training  resources  and  activities  on  both 
prescriptive  and  empirical  measures  of  readiness.  The  data  base  options  and  strategies 
developed  within  the  TMM's  functional  guidelines  are  summarized  in  Table  6  as  a  Macro- 
Level  Training  Data  Base  Master  Plan. 

The  proposed  Macro-Level  Training  Data  Master  Plan  identifies  the  types  of  data 
that  the  macro-level  training  data  bases  should  contain  and  provides  a  road  map  for  data 
base  development  with  near-term,  intermediate,  and  long-term  objectives.  The  Plan 
provides  a  vehicle  forjudging  the  adequacy  of  the  current  Macro-Level  Cost  Data  Bases  by 
not  only  providing  guidance  on  what  they  should  ideally  contain  but  also  by  providing 
guidance  on  the  kinds  of  data  that  they  might  reasonably  be  expected  to  contain. 

It  should  be  noted  that  recommended  data  and  data  base  development  actions 
represent  the  TMM's  projection  of  what  the  macro-level  training  manager  should  seek  to 
develop  as  a  data  base  that  would  support  a  wide  range  of  policymaking  activities.  The 
Plan  does  not  directly  address  either  the  technical  or  administrative  feasibility  of  acquiring 
all  of  the  necessary  information.  Generally  the  more  future  oriented  items  will  also  be  the 
most  technologically  and/or  administratively  difficult  to  achieve.  However,  the 
accomplishment  of  any  of  the  data  base  development  goals  will  enhance  the  training 
manager's  policymaking  capabilities  and  general  ability  to  compete  for  scarce  resources. 

The  establishment  of  data  bases  to  support  the  near-term  goals  of  the  Cost- 
Production  function  should  receive  the  highest  priority: 
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Table  6.  Macro-Level  Training  Data  Base  Master  Plan 


TMM  Functions/ 
Types  of  Data 

Near-Term 
(1 -2  yrs) 

Intermediate 
(3-5  yrs) 

Long-Term 
(over  5  yrs) 

COST- 

PRODUCTION 

Data  types: 

1 .  Cost 

2.  Effectiveness 

3.  Cost-Effectiveness 

4.  Training  System 

Capabilities, 
Capacities,  and 
Utilization  Rates 

Develop  descriptive 
cost- effectiveness  data 
bases  capable  of 
tracking  and  summarizing 
budget  allocations  for 
training,  training  costs, 
student  loads,  and 
inventory  projections. 

Develop  descriptive 
training  system 
management  data  bases 
capable  of  summarizing 
and  simplifying  the 
available  information 
about  training  capabilities, 
capacities,  locations,  and 
the  levels  of  utilization  of 
the  training  facilities, 
ranges,  and  equipment. 

Develop  census  types 
of  trend  data  by 
incrementing  the  data 
bases  on  an  annual 
basis. 

Relate  past  levels  of 
funding  to  current 
status  and  project 
current  levels  of 
funding  to  the  future 
status  of  the  training 
system. 

Develop  and  verify  key 
functional  relationships 
which  can  be  used  to 
predict  changes  in  the 
training  system  or  the 
cost-effectiveness  of 
changes  in  resources, 
personnel,  or  of 
training  technology. 

Phase  1 :  Summarize 
functional  relationships 
obtained  from  the  data 
bases  and  from  the 
research  literature  into 
a  Training  Management 
Information  Dictionary. 

Phase  2:  Build  a  data 
base  collection  of 
functional  relationships 
that  is  sufficient  to  be 
able  to  simulate  the 
operation  of  the  training 
system  or  at  least  large 
parts  of  it. 

(Continued) 
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Table  6  (cont’d).  Macro-Level  Training  Data  Base  Master  Plan 


TMM  Functions/ 
Types  of  Data 

Near-Term 
(1 -2  yrs) 

Intermediate 
(3-5  yrs) 

Long-Term 
(over  5  yrs) 

PERFORMANCE 

EFFECTIVENESS 

Data  Types: 

Performance  quality 
measures  obtained 
during  and  after 
training  events  which 
can  be  attributed  to 
the  training  received 
by  individuals  and 
units. 

Develop  training  data 
bases  consisting  of 
individual  selection  data, 
training  histories,  and 
operational  achievement 
data  in  order  to  establish 
a  historical  baseline  that 
would  support  analyses 
of  the  effects  of  changes 
in  the  training  program. 

Extend  the  methodolo¬ 
gies  used  to  assess  the 
cost-effectiveness  of 
flight  simulators  to  other 
simulation  and  training 
technology  evaluations 
as  a  first  step  in 
establishing  guidelines 
for  the  routine  collection 
of  cost-effectiveness 
data 

Establish  a  system 
that  would  require  the 
production  of  Net 
Productivity  Ratings 
on  all  first-term 
personnel  assigned 
to  the  command  to 
provide  a  uniform 
measure  of  training 
effectiveness. 

Collect  data  relating 
post-initial-skills 
training  activities  to 
the  maintenance  or 
improvement  of  skills 
while  performing  in 
an  operational 
environment. 

Develop  performance 
measures  that  relate 
training  achievement 
levels  to  operationally 
meaningful  skill 
levels. 

Develop  and  validate 
highly  aggregated 
measures  of  the 
effects  of  collective 
training  (e.g.,  collective 
training  as  a  force 
multiplier). 

(Continued) 
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Table  6  (cont’d).  Macro-Level  Training  Data  Base  Master  Plan 


TMM  Functions/ 
Types  of  Data 

Near-Term 
(1  -2  yrs) 

Intermediate 
(3-5  yrs) 

Long-Term 
(over  5  yrs) 

READINESS 

EVALUATION 

Data  Types: 

Prescriptive-data  used 
to  verify  that  approved 
resources  have 
provided  the  required 
numbers  of  trained 
personnel  and  that 
both  the  personnel 
and  the  operational 
units  have  received  the 
prescribed  courses 
and  training  events. 

Empirical-data  used 
to  evaluate  the  effects 
of  training  on  the 
ability  of  an  operational 
unit  to  perform  its 
assigned  mission  or  to 
perform  mission-related 
tasks. 

Collect  and  summarize 
existing  information  on 
the  relationship  between 
training  and  performance 
in  combat  or  combat-like 
situations.  This  will 
enable  the  training 
manager  to  support 
training  policy  decisions 
with  instances  in  which 
training  changes  made  a 
significant  impact;  e.g., 
the  change  in  kill  ratios 
after  the  establishment 
of  the  Navy  Fighter 
Weapons  School. 

Concentrate  on 
collecting  prescriptive 
data  relating  training 
budgets  to  the  levels 
of  training  accom¬ 
plished  in  terms  of  the 
levels  of  training 
required. 

Relate  training  budgets 
to  the  training  related 
aspects  of  readiness 
overtime  to  provide  a 
measure  of  the  effects 
of  budgets  on  the 
training  system's  ability 
to  fulfill  its  mission  and 
define  the  nature  of 
the  functional  relation¬ 
ship  between  training 
budgets  and  readiness. 

Develop  better  ways  to 
assess  the  effects  of 
advanced  collective 
training  efforts  such  as 
simulated  combat  training 
on  instrumented  ranges, 
networked  simulations, 
and  war  games.  Use 
these  data  to  establish 
the  necessary  linkages 
between  training  and 
needed  combat 
capabilities. 
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(1)  Develop  descriptive  cost-effectiveness  data  bases  capable  of  tracking  and 
summarizing  budget  allocations  for  training,  training  costs,  student  loads,  and 
inventory  projections. 

(2)  Develop  descriptive  training  system  management  data  bases  capable  of 
summarizing  and  simplifying  the  available  information  about  training 
capabilities,  capacities,  locations  and  the  levels  of  utilization  of  the  training 
facilities,  ranges,  and  equipment 

-^commendation  is  based  on  the  importance  of  these  kinds  of  data  to  the  every-day 
functioning  of  the  macro-level  training  manager  and  the  fact  that  they  should  generally  be 
more  available  for  collection  and  inclusion  in  a  data  base. 
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